You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Fariborz Navidan <md...@gmail.com> on 2019/08/28 11:28:42 UTC
Unable to start System VMs
Hello,
New ACS management is unable to start System VMs. Below are logs. ACS is
still poor in error handling and reporting. It should be able to retyrn
clear error messages inside UI.
2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
expired async-jobs
2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup expired
async-jobs
2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
expired async-jobs
2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup expired
async-jobs
2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566988128964
2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) ===END===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566988128964
2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START=== 137.74.35.102
-- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START=== 137.74.35.102
-- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128967
2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
(qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api Discovery
plugin was unable to find an api by that name or process any apis
2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566988128967
2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
expired async-jobs
2019-08-28 13:22:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers to
update status.
2019-08-28 13:22:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC networks
to update Redundant State.
2019-08-28 13:22:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 networks to
update RvR status.
2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup expired
async-jobs
2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) ===END===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START=== 137.74.35.102
-- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START=== 137.74.35.102
-- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128971
2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
(qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api Discovery
plugin was unable to find an api by that name or process any apis
2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566988128971
2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for reconnect
2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts suitable for
reconnect
2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters already
owned by this management server
2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for clusters
already owned by this management server
2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not owned
by any management server
2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for clusters
not owned by any management server
2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128972
2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128972
2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
expired async-jobs
2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup expired
async-jobs
2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
number of records in report: 0
2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Done with process of VM state
report. host: 1
2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor is
running...
2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128973
2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128973
2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector is
running...
2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
(StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq 1-5778962746846478713:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
running...
2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no secondary
storage VM for secondary storage host Secondary1
2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
default host to execute commands because the command is not of Copy type.
2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq 1-5778962746846478714:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
expired async-jobs
2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup expired
async-jobs
2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to launch
secondary storage VM
2019-08-28 13:22:40,562 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary storage
vms found in datacenter id=1, starting one
2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary storage
vm is available, need to allocate a new secondary storage vm
2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage vm
from a newly started instance for request from data center : 1
2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
configuration for offering [Network Offering
[6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
configuration for offering [Network Offering
[3-Control-System-Control-Network]: Ntwk[201|Control|3]
2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
configuration for offering [Network Offering
[2-Management-System-Management-Network]: Ntwk[200|Management|2]
2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to launch
console proxy
2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Expand console proxy standby
capacity for zone France
2019-08-28 13:22:40,615 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) No stopped console proxy is
available, need to allocate a new console proxy
2019-08-28 13:22:40,616 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Assign console proxy from a
newly started instance for request from data center : 1
2019-08-28 13:22:40,623 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating entries for VM:
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,625 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nics for
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,628 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
VM[SecondaryStorageVm|s-67-VM] in network Ntwk[203|Guest|6] with requested
profile NicProfile[0-0-null-null-null
2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
configuration for offering [Network Offering
[6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
configuration for offering [Network Offering
[3-Control-System-Control-Network]: Ntwk[201|Control|3]
2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
configuration for offering [Network Offering
[2-Management-System-Management-Network]: Ntwk[200|Management|2]
2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,660 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
VM[SecondaryStorageVm|s-67-VM] in network Ntwk[201|Control|3] with
requested profile null
2019-08-28 13:22:40,668 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
VM[SecondaryStorageVm|s-67-VM] in network Ntwk[200|Management|2] with
requested profile null
2019-08-28 13:22:40,673 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating entries for VM:
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,675 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nics for
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,676 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
VM[ConsoleProxy|v-68-VM] in network Ntwk[203|Guest|6] with requested
profile NicProfile[0-0-null-null-null
2019-08-28 13:22:40,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating disks for
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,681 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocation completed for VM:
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
alert
2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) New secondary storage vm
created, zone: France, secStorageVm: s-67-VM, public IP: null, private IP:
null
2019-08-28 13:22:40,688 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
VM[ConsoleProxy|v-68-VM] in network Ntwk[201|Control|3] with requested
profile null
2019-08-28 13:22:40,690 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Sync job-218 execution on
object VmWorkJobQueue.67
2019-08-28 13:22:40,691 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
VM[ConsoleProxy|v-68-VM] in network Ntwk[200|Management|2] with requested
profile null
2019-08-28 13:22:40,694 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating disks for
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,696 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocation completed for VM:
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy alert
2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) New console proxy created,
zone: France, proxy: v-68-VM, public IP: null, private IP: null
2019-08-28 13:22:40,701 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Sync job-219 execution on
object VmWorkJobQueue.68
2019-08-28 13:22:40,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
item: SyncQueueItemVO {id:194, queueId: 191, contentType: AsyncJob,
contentId: 218, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
13:22:40 CEST 2019}
2019-08-28 13:22:40,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
job-218
2019-08-28 13:22:40,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
item: SyncQueueItemVO {id:195, queueId: 192, contentType: AsyncJob,
contentId: 219, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
13:22:40 CEST 2019}
2019-08-28 13:22:40,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
job-219
2019-08-28 13:22:40,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:d5030cfc) Add
job-218 into job monitoring
2019-08-28 13:22:40,872 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:8371f7e8) Add
job-219 into job monitoring
2019-08-28 13:22:40,877 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
Executing AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}
2019-08-28 13:22:40,877 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Run VM
work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:22:40,878 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
Executing AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}
2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Run VM
work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:22:40,880 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:22:40,880 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Successfully transitioned to start state for
VM[SecondaryStorageVm|s-67-VM] reservation id =
c608b88c-7d06-411e-bb84-f2d5e2b3390d
2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,881 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Successfully transitioned to start state for
VM[ConsoleProxy|v-68-VM] reservation id =
6c836a9c-0d16-42c3-86c7-446febb1a189
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:40,882 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching all possible resources under this Zone: 1
2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching all possible resources under this Zone: 1
2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:40,886 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:40,886 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:40,887 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 536870912 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
Requested RAM: 536870912
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
available
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding to
list: 1
2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
suitable hosts
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) We need to allocate new storagepool for this volume
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Calling StoragePoolAllocators to find suitable pools
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 1073741824 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) System VMs will use shared storage for zone id=1
2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) LocalStoragePoolAllocator trying to find storage pool to
fit the vm
2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) ClusterScopeStoragePoolAllocator looking for storage pool
2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
pools will be ignored.
2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Found pools matching tags: [Pool[1|SharedMountPoint]]
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
Requested RAM: 1073741824
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
available
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding to
list: 1
2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
suitable hosts
2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking if storage pool is suitable, name: null ,poolId: 1
2019-08-28 13:22:40,891 INFO [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Storage pool null (1) does not supply IOPS capacity,
assuming enough capacity
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) We need to allocate new storagepool for this volume
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Calling StoragePoolAllocators to find suitable pools
2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking pool 1 for storage, totalSize: 984895062016,
usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold: 0.85
2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Destination pool id: 1
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) System VMs will use shared storage for zone id=1
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) LocalStoragePoolAllocator trying to find storage pool to
fit the vm
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) ClusterScopeStoragePoolAllocator looking for storage pool
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
pools will be ignored.
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Found pools matching tags: [Pool[1|SharedMountPoint]]
2019-08-28 13:22:40,893 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking if storage pool is suitable, name: null ,poolId: 1
2019-08-28 13:22:40,893 INFO [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Storage pool null (1) does not supply IOPS capacity,
assuming enough capacity
2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking pool 1 for storage, totalSize: 984895062016,
usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold: 0.85
2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Destination pool id: 1
2019-08-28 13:22:40,896 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Pool ID for the volume with ID 62 is null
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Found storage pool Primary1 of type SharedMountPoint
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Total capacity of the pool Primary1 with ID 1 is
984895062016
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking pool: 1 for storage allocation , maxSize :
984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
disable threshold: 0.85
2019-08-28 13:22:40,897 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) ClusterScopeStoragePoolAllocator returning 1 suitable
storage pools
2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Host: 1 can access pool: 1
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Pool ID for the volume with ID 63 is null
2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Found storage pool Primary1 of type SharedMountPoint
2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Total capacity of the pool Primary1 with ID 1 is
984895062016
2019-08-28 13:22:40,898 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:22:40,899 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking pool: 1 for storage allocation , maxSize :
984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
disable threshold: 0.85
2019-08-28 13:22:40,899 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) ClusterScopeStoragePoolAllocator returning 1 suitable
storage pools
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Host: 1 can access pool: 1
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested RAM:
536870912
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
used:500, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
requested mem: 536870912,alloc_from_last:false
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU: 500
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
,Requested RAM: 1073741824
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
reserved: 0; requested mem: 1073741824,alloc_from_last:false
2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Network id=203 is already implemented
2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Network id=203 is already implemented
2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Asking VirtualRouter to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Asking VirtualRouter to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Asking SecurityGroupProvider to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:22:40,913 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Asking SecurityGroupProvider to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:22:40,914 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Network id=201 is already implemented
2019-08-28 13:22:40,915 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Network id=201 is already implemented
2019-08-28 13:22:40,919 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Network id=200 is already implemented
2019-08-28 13:22:40,920 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Network id=200 is already implemented
2019-08-28 13:22:40,922 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Releasing ip address for instance=186
2019-08-28 13:22:40,923 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Releasing ip address for instance=189
2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Allocated a nic
NicProfile[186-67-c608b88c-7d06-411e-bb84-f2d5e2b3390d-192.168.0.226-null
for VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Allocated a nic
NicProfile[189-68-6c836a9c-0d16-42c3-86c7-446febb1a189-192.168.0.121-null
for VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) template 3 is already in store:1, type:Image
2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) template 3 is already in store:1, type:Image
2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) template 3 is already in store:1, type:Primary
2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) template 3 is already in store:1, type:Primary
2019-08-28 13:22:40,932 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) copyAsync inspecting src type TEMPLATE copyAsync
inspecting dest type VOLUME
2019-08-28 13:22:40,933 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) copyAsync inspecting src type TEMPLATE copyAsync
inspecting dest type VOLUME
2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) getCommandHostDelegation: class
org.apache.cloudstack.storage.command.CopyCommand
2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) We are returning the default host to execute commands
because the target hypervisor of the source data is not XenServer.
2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) getCommandHostDelegation: class
org.apache.cloudstack.storage.command.CopyCommand
2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) We are returning the default host to execute commands
because the target hypervisor of the source data is not XenServer.
2019-08-28 13:22:40,934 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478715: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
Template
(KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
}
2019-08-28 13:22:40,935 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478716: Waiting for Seq
5778962746846478715 Scheduling: { Cmd , MgmtId: 205425461278937, via: 1(
fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
Template
(KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
}
2019-08-28 13:22:41,037 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-138d8f65) (logid:85352d35) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128974
2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478715:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
}
2019-08-28 13:22:41,037 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478716: Sending
now. is current sequence.
2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478715: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
CopyCmdAnswer } }
2019-08-28 13:22:41,040 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:41,047 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128974
2019-08-28 13:22:41,057 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
2019-08-28 13:22:41,071 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478717: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
}
2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
}
2019-08-28 13:22:41,126 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716: No more
commands found
2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478716: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
CopyCmdAnswer } }
2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-9:null) (logid:) Seq 1-5778962746846478717:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478717: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:22:41,128 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:22:41,129 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Cleaning up resources for the vm
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:22:41,131 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478718: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.64","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:22:41,139 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.2.191
eth0mask=255.255.0.0 eth1ip=192.168.0.121 eth1mask=255.255.255.0 mgmtcidr=
51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
2019-08-28 13:22:41,151 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478719: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.191","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
}
2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-8:null) (logid:) Seq 1-5778962746846478719:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478719: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:22:41,323 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:22:41,326 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
in Starting state
2019-08-28 13:22:41,337 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478720: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.2.191","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:22:44,313 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-10:null) (logid:) Seq 1-5778962746846478718:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:22:44,314 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478718: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:22:44,318 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
2019-08-28 13:22:44,321 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Releasing ip address for
reservationId=c608b88c-7d06-411e-bb84-f2d5e2b3390d, instance=186
2019-08-28 13:22:44,321 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Successfully released network resources for the vm
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Successfully cleaned up resources for the VM
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:22:44,324 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:22:44,324 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:44,327 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:44,328 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
suitable hosts
2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) No suitable hosts found
2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) No suitable hosts found under this Cluster: 1
2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) The specified cluster is in avoid set, returning.
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:22:44,331 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching all possible resources under this Zone: 1
2019-08-28 13:22:44,332 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:44,333 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:22:44,334 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:22:44,338 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:22:44,343 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,343 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,343 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:22:44,344 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Unable
to complete AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}, job origin:76
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,344 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
Complete async job-218, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
Publish async job-218 complete on message bus
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
up jobs related to job-218
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Update
db status for job-218
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
up jobs joined with job-218 and disjoin all subjobs created from job- 218
2019-08-28 13:22:44,349 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
executing com.cloud.vm.VmWorkStart for job-218
2019-08-28 13:22:44,350 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Remove
job-218 from job monitoring
2019-08-28 13:22:44,351 WARN [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Exception while trying to
start secondary storage vm
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,352 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Unable to start secondary
storage vm for standby capacity, vm id : 67, will recycle it and start a
new one
2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
alert
2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Secondary Storage Vm creation
failure, zone: France
2019-08-28 13:22:44,352 WARN [c.c.a.AlertManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) AlertType:: 19 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
Storage Vm creation failure. zone: France, error details: null
2019-08-28 13:22:44,353 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Primary secondary storage is
not even started, wait until next turn
2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-11:null) (logid:) Seq 1-5778962746846478720:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478720: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:22:44,436 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
2019-08-28 13:22:44,438 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Releasing ip address for
reservationId=6c836a9c-0d16-42c3-86c7-446febb1a189, instance=189
2019-08-28 13:22:44,438 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Successfully released network resources for the vm
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Successfully cleaned up resources for the VM
VM[ConsoleProxy|v-68-VM] in Starting state
2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:22:44,441 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:44,443 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
suitable hosts
2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) No suitable hosts found
2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) No suitable hosts found under this Cluster: 1
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) The specified cluster is in avoid set, returning.
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching all possible resources under this Zone: 1
2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:22:44,450 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,453 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,453 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Unable
to complete AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}, job origin:77
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
Complete async job-219, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
Publish async job-219 complete on message bus
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
up jobs related to job-219
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Update
db status for job-219
2019-08-28 13:22:44,455 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
up jobs joined with job-219 and disjoin all subjobs created from job- 219
2019-08-28 13:22:44,457 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
executing com.cloud.vm.VmWorkStart for job-219
2019-08-28 13:22:44,458 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Remove
job-219 from job monitoring
2019-08-28 13:22:44,459 WARN [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Exception while trying to
start console proxy
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,460 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Unable to start console
proxy vm for standby capacity, vm id : 68, will recycle it and start a new
one
2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy alert
2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Console proxy creation
failure, zone: France
2019-08-28 13:22:44,460 WARN [c.c.a.AlertManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) AlertType:: 10 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
proxy creation failure. zone: France, error details: null
2019-08-28 13:22:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) Begin cleanup
expired async-jobs
2019-08-28 13:22:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 routers to
update status.
2019-08-28 13:22:44,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 VPC networks
to update Redundant State.
2019-08-28 13:22:44,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 networks to
update RvR status.
2019-08-28 13:22:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) End cleanup expired
async-jobs
2019-08-28 13:22:48,876 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-1ca6b8a4) (logid:767cda38) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128975
2019-08-28 13:22:48,883 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:48,903 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128975
2019-08-28 13:22:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) Begin cleanup
expired async-jobs
2019-08-28 13:22:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) End cleanup expired
async-jobs
2019-08-28 13:22:56,741 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-3fa512ab) (logid:4609e172) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128976
2019-08-28 13:22:56,747 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:56,751 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128976
2019-08-28 13:23:03,063 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a0c69e67) (logid:0a2b6c35) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128977
2019-08-28 13:23:03,071 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:23:03,091 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128977
2019-08-28 13:23:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) Begin cleanup
expired async-jobs
2019-08-28 13:23:04,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) End cleanup expired
async-jobs
2019-08-28 13:23:09,900 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-16:ctx-9f3738f2) (logid:dea95b1b) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128978
2019-08-28 13:23:09,907 DEBUG [c.c.a.ApiServer]
(qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:23:09,929 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128978
2019-08-28 13:23:10,557 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Zone 1 is ready to launch
secondary storage VM
2019-08-28 13:23:10,561 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) No running secondary storage
vms found in datacenter id=1, starting one
2019-08-28 13:23:10,570 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Found a stopped secondary
storage vm, starting it. Vm id : 67
2019-08-28 13:23:10,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-220 execution on
object VmWorkJobQueue.67
2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Zone 1 is ready to launch
console proxy
2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expand console proxy standby
capacity for zone France
2019-08-28 13:23:10,616 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Found a stopped console
proxy, starting it. Vm id : 68
2019-08-28 13:23:10,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-221 execution on
object VmWorkJobQueue.68
2019-08-28 13:23:10,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
item: SyncQueueItemVO {id:196, queueId: 191, contentType: AsyncJob,
contentId: 220, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
13:23:10 CEST 2019}
2019-08-28 13:23:10,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
job-220
2019-08-28 13:23:10,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
item: SyncQueueItemVO {id:197, queueId: 192, contentType: AsyncJob,
contentId: 221, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
13:23:10 CEST 2019}
2019-08-28 13:23:10,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
job-221
2019-08-28 13:23:10,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:f78794fa) Add
job-220 into job monitoring
2019-08-28 13:23:10,871 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:7deee6cf) Add
job-221 into job monitoring
2019-08-28 13:23:10,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
Executing AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}
2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Run VM
work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:10,881 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
Executing AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}
2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Run VM
work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:10,883 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:23:10,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Successfully transitioned to start state for
VM[SecondaryStorageVm|s-67-VM] reservation id =
46a0e954-a9af-45af-ac39-91e036654f95
2019-08-28 13:23:10,884 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Successfully transitioned to start state for
VM[ConsoleProxy|v-68-VM] reservation id =
a25c0118-b49c-41b2-8970-9b73191746f3
2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:10,885 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,886 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 1073741824 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 536870912 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
Requested RAM: 1073741824
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
available
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding to
list: 1
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
suitable hosts
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
Requested RAM: 536870912
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
available
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding to
list: 1
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
suitable hosts
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Volume has pool already allocated, checking if pool can be
reused, poolId: 1
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Volume has pool already allocated, checking if pool can be
reused, poolId: 1
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Planner need not allocate a pool for this volume since its
READY
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Planner need not allocate a pool for this volume since its
READY
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Host: 1 can access pool: 1
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Host: 1 can access pool: 1
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested RAM:
536870912
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
used:500, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
requested mem: 536870912,alloc_from_last:false
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU: 500
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
,Requested RAM: 1073741824
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
reserved: 0; requested mem: 1073741824,alloc_from_last:false
2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Network id=203 is already implemented
2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Network id=203 is already implemented
2019-08-28 13:23:10,907 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Asking VirtualRouter to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Asking VirtualRouter to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Asking SecurityGroupProvider to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:10,909 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Asking SecurityGroupProvider to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:10,911 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Network id=201 is already implemented
2019-08-28 13:23:10,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Network id=201 is already implemented
2019-08-28 13:23:10,916 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Network id=200 is already implemented
2019-08-28 13:23:10,918 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Network id=200 is already implemented
2019-08-28 13:23:10,918 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Releasing ip address for instance=186
2019-08-28 13:23:10,920 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Allocated a nic
NicProfile[186-67-46a0e954-a9af-45af-ac39-91e036654f95-192.168.0.86-null
for VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:10,921 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Releasing ip address for instance=189
2019-08-28 13:23:10,922 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No need to recreate the volume: Vol[62|vm=67|ROOT], since
it already has a pool assigned: 1, adding disk to VM
2019-08-28 13:23:10,923 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Allocated a nic
NicProfile[189-68-a25c0118-b49c-41b2-8970-9b73191746f3-192.168.0.20-null
for VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:10,925 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No need to recreate the volume: Vol[63|vm=68|ROOT], since
it already has a pool assigned: 1, adding disk to VM
2019-08-28 13:23:10,930 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
2019-08-28 13:23:10,931 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.0.248
eth0mask=255.255.0.0 eth1ip=192.168.0.20 eth1mask=255.255.255.0 mgmtcidr=
51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
2019-08-28 13:23:10,941 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478721: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.170","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
}
2019-08-28 13:23:10,942 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478722: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.248","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
}
2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-12:null) (logid:) Seq 1-5778962746846478722:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478722: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:23:10,985 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:23:10,986 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
in Starting state
2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478723: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.0.248","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-13:null) (logid:) Seq 1-5778962746846478721:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478721: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:23:10,990 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Cleaning up resources for the vm
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:23:10,991 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478724: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.170","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:23:14,130 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-14:null) (logid:) Seq 1-5778962746846478724:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,131 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478724: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:23:14,136 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
2019-08-28 13:23:14,138 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Releasing ip address for
reservationId=46a0e954-a9af-45af-ac39-91e036654f95, instance=186
2019-08-28 13:23:14,139 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Successfully released network resources for the vm
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Successfully cleaned up resources for the VM
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:23:14,142 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,145 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:14,145 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:14,146 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
suitable hosts
2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No suitable hosts found
2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No suitable hosts found under this Cluster: 1
2019-08-28 13:23:14,148 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) The specified cluster is in avoid set, returning.
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:null, requested cpu: 500, requested ram: 536870912
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Pod: 1
2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Pod: 1
2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:23:14,156 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:23:14,160 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,161 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,161 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:23:14,161 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Unable
to complete AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}, job origin:76
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
Complete async job-220, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
Publish async job-220 complete on message bus
2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
up jobs related to job-220
2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Update
db status for job-220
2019-08-28 13:23:14,163 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
up jobs joined with job-220 and disjoin all subjobs created from job- 220
2019-08-28 13:23:14,166 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
executing com.cloud.vm.VmWorkStart for job-220
2019-08-28 13:23:14,166 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Remove
job-220 from job monitoring
2019-08-28 13:23:14,169 WARN [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Exception while trying to
start secondary storage vm
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,169 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Unable to start secondary
storage vm for standby capacity, vm id : 67, will recycle it and start a
new one
2019-08-28 13:23:14,172 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-222 execution on
object VmWorkJobQueue.67
2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-15:null) (logid:) Seq 1-5778962746846478723:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478723: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:23:14,623 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
2019-08-28 13:23:14,626 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Releasing ip address for
reservationId=a25c0118-b49c-41b2-8970-9b73191746f3, instance=189
2019-08-28 13:23:14,626 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Successfully released network resources for the vm
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Successfully cleaned up resources for the VM
VM[ConsoleProxy|v-68-VM] in Starting state
2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:23:14,631 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,635 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:14,635 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:14,636 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
suitable hosts
2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No suitable hosts found
2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No suitable hosts found under this Cluster: 1
2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:23:14,638 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,639 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) The specified cluster is in avoid set, returning.
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:null, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Pod: 1
2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Pod: 1
2019-08-28 13:23:14,641 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:23:14,642 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:23:14,646 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:23:14,650 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,651 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,651 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:23:14,651 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Unable
to complete AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}, job origin:77
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
Complete async job-221, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
Publish async job-221 complete on message bus
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
up jobs related to job-221
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Update
db status for job-221
2019-08-28 13:23:14,653 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
up jobs joined with job-221 and disjoin all subjobs created from job- 221
2019-08-28 13:23:14,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
executing com.cloud.vm.VmWorkStart for job-221
2019-08-28 13:23:14,656 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Remove
job-221 from job monitoring
2019-08-28 13:23:14,658 WARN [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Exception while trying to
start console proxy
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,658 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Unable to start console
proxy vm for standby capacity, vm id : 68, will recycle it and start a new
one
2019-08-28 13:23:14,662 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-223 execution on
object VmWorkJobQueue.68
2019-08-28 13:23:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 routers to
update status.
2019-08-28 13:23:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 VPC networks
to update Redundant State.
2019-08-28 13:23:14,855 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
item: SyncQueueItemVO {id:198, queueId: 191, contentType: AsyncJob,
contentId: 222, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
13:23:14 CEST 2019}
2019-08-28 13:23:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 networks to
update RvR status.
2019-08-28 13:23:14,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
job-222
2019-08-28 13:23:14,859 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
item: SyncQueueItemVO {id:199, queueId: 192, contentType: AsyncJob,
contentId: 223, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
13:23:14 CEST 2019}
2019-08-28 13:23:14,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
job-223
2019-08-28 13:23:14,863 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:7e4a2784) Add
job-222 into job monitoring
2019-08-28 13:23:14,869 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:26933ca4) Add
job-223 into job monitoring
2019-08-28 13:23:14,871 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) Begin cleanup
expired async-jobs
2019-08-28 13:23:14,875 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
Executing AsyncJobVO {id:222, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
null}
2019-08-28 13:23:14,875 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Run VM
work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
2019-08-28 13:23:14,877 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,879 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) End cleanup expired
async-jobs
2019-08-28 13:23:14,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
Executing AsyncJobVO {id:223, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
null}
2019-08-28 13:23:14,881 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Run VM
work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
2019-08-28 13:23:14,882 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) VM is already stopped: VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,883 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,883 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Complete async job-222, jobStatus: SUCCEEDED, resultCode:
0, result: null
2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Publish async job-222 complete on message bus
2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Wake up jobs related to job-222
2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Update db status for job-222
2019-08-28 13:23:14,886 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Wake up jobs joined with job-222 and disjoin all subjobs
created from job- 222
2019-08-28 13:23:14,888 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) VM is already stopped: VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,888 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,888 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Complete async job-223, jobStatus: SUCCEEDED, resultCode:
0, result: null
2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Publish async job-223 complete on message bus
2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Wake up jobs related to job-223
2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Update db status for job-223
2019-08-28 13:23:14,892 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Wake up jobs joined with job-223 and disjoin all subjobs
created from job- 223
2019-08-28 13:23:14,895 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
2019-08-28 13:23:14,895 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
executing com.cloud.vm.VmWorkStop for job-222
2019-08-28 13:23:14,898 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Remove
job-222 from job monitoring
2019-08-28 13:23:14,901 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
2019-08-28 13:23:14,901 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
executing com.cloud.vm.VmWorkStop for job-223
2019-08-28 13:23:14,903 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Remove
job-223 from job monitoring
2019-08-28 13:23:14,916 DEBUG [c.c.c.CapacityManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) VM state transitted from
:Stopped to Expunging with event: ExpungeOperationvm's original host id:
null new host id: null host id before state transition: null
2019-08-28 13:23:14,921 DEBUG [c.c.c.CapacityManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) VM state transitted from
:Stopped to Expunging with event: ExpungeOperationvm's original host id:
null new host id: null host id before state transition: null
2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Destroying vm
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up NICS
2019-08-28 13:23:14,923 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning network for vm: 67
2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Destroying vm
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up NICS
2019-08-28 13:23:14,929 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning network for vm: 68
2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking VirtualRouter to
release Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking SecurityGroupProvider
to release Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:14,936 DEBUG [c.c.n.g.DirectNetworkGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Deallocate network: networkId:
203, ip: 137.74.82.208
2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking VirtualRouter to
release Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking SecurityGroupProvider
to release Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Deallocate network:
networkId: 203, ip: 137.74.82.209
2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) remove nic 184 secondary ip
2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=184
2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,943 DEBUG [c.c.n.g.DirectNetworkGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) remove nic 187 secondary ip
2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=187
2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=185
2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,949 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=188
2019-08-28 13:23:14,950 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=186
2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,952 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up hypervisor data
structures (ex. SRs in XenServer) for managed storage
2019-08-28 13:23:14,952 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning storage for vm: 67
2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=189
2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,953 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up hypervisor data
structures (ex. SRs in XenServer) for managed storage
2019-08-28 13:23:14,954 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning storage for vm: 68
2019-08-28 13:23:14,964 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) getCommandHostDelegation:
class org.apache.cloudstack.storage.command.DeleteCommand
2019-08-28 13:23:14,964 DEBUG [c.c.h.XenServerGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) We are returning the default
host to execute commands because the command is not of Copy type.
2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-59a5da7a)
(logid:2183e74c) Seq 1-5778962746846478725: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
}
2019-08-28 13:23:14,965 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) getCommandHostDelegation:
class org.apache.cloudstack.storage.command.DeleteCommand
2019-08-28 13:23:14,965 DEBUG [c.c.h.XenServerGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) We are returning the default
host to execute commands because the command is not of Copy type.
2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
Sending { Cmd , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com), Ver:
v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
}
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-2:null) (logid:) Seq 1-5778962746846478726:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-1:null) (logid:) Seq 1-5778962746846478725:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { Answer } }
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-59a5da7a)
(logid:2183e74c) Seq 1-5778962746846478725: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, { Answer }
}
2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Volume 63 is not referred
anywhere, remove it from volumes table
2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Volume 62 is not referred
anywhere, remove it from volumes table
2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expunged
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Expunged
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) received secondary storage vm
alert
2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Secondary Storage Vm creation
failure, zone: France
2019-08-28 13:23:14,975 WARN [c.c.a.AlertManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) AlertType:: 19 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
Storage Vm creation failure. zone: France, error details: null
2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) received console proxy alert
2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Console proxy creation
failure, zone: France
2019-08-28 13:23:14,976 WARN [c.c.a.AlertManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) AlertType:: 10 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
proxy creation failure. zone: France, error details: null
2019-08-28 13:23:14,977 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Primary secondary storage is
not even started, wait until next turn
2019-08-28 13:23:16,259 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-bd2c1e74) (logid:dc193f19) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128979
2019-08-28 13:23:16,268 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:23:16,286 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128979
2019-08-28 13:23:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) Begin cleanup
expired async-jobs
2019-08-28 13:23:24,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) End cleanup expired
async-jobs
2019-08-28 13:23:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
number of records in report: 0
2019-08-28 13:23:24,958 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Done with process of VM state
report. host: 1
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
Hello,
Simply went for a full re-install and System VMs and VR is now up and
running. However now I have a networking issue such that VMs cannot reach
the internet. System VMs cannot reach the guest network gateway.
Communication between Management server (the same as KVM host) and VMs is
OK. Please review my network config bellow and guide me to fix the issue.
*Inside console proxy:*
root@v-16-VM:~# ping 137.74.82.222
PING 137.74.82.222 (137.74.82.222): 56 data bytes
92 bytes from v-16-VM (137.74.82.209): Destination Host Unreachable
*On the KVM host:*
eth0: Not used for cloud
eth1: vRack interface on OVH network (Private network with public internet
routing capability, IP blocks are routed to this virtual network)
Guest traffic label: cloudbr0
Management traffic label: cloudbr0
Storage traffic label: cloudbr0 (Storage is on local NFS)
Guest VLAN ID: 200 (Not sure if it is arbitrary or should match with host)
[root@fr-kvm1 network-scripts]# brctl show
bridge name bridge id STP enabled interfaces
breth1-200 8000.d05099d38682 no eth1.200
vnet2
vnet5
vnet6
cloud0 8000.fe00a9fe004c no vnet0
vnet3
vnet7
cloudbr0 8000.d05099d38682 yes eth1
vnet1
vnet4
cloudbr1 8000.d05099d38682 yes eth1.100
virbr0 8000.5254007c0a0f yes virbr0-nic
[root@fr-kvm1 network-scripts]#
I have also tested the IP block routing by bringing up an alias interface
on eth1 and it works. I have also enabled net. Also below is long list of
iptables rules:
[root@fr-kvm1 network-scripts]# iptables --list
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
Chain FORWARD (policy ACCEPT)
target prot opt source destination
BF-breth1-200 all -- anywhere anywhere PHYSDEV
match --physdev-is-bridged
BF-breth1-200 all -- anywhere anywhere PHYSDEV
match --physdev-is-bridged
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
BF-cloudbr0 all -- anywhere anywhere PHYSDEV
match --physdev-is-bridged
BF-cloudbr0 all -- anywhere anywhere PHYSDEV
match --physdev-is-bridged
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
ACCEPT all -- anywhere 192.168.122.0/24 ctstate
RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with
icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with
icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:bootpc
Chain BF-breth1-200 (2 references)
target prot opt source destination
Chain BF-breth1-200-IN (0 references)
target prot opt source destination
r-3-VM all -- anywhere anywhere PHYSDEV match
--physdev-in vnet6 --physdev-is-bridged
Chain BF-breth1-200-OUT (0 references)
target prot opt source destination
r-3-VM all -- anywhere anywhere PHYSDEV match
--physdev-out vnet6 --physdev-is-bridged
Chain BF-cloudbr0 (2 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere state
RELATED,ESTABLISHED
BF-cloudbr0-IN all -- anywhere anywhere PHYSDEV
match --physdev-is-in --physdev-is-bridged
BF-cloudbr0-OUT all -- anywhere anywhere PHYSDEV
match --physdev-is-out --physdev-is-bridged
ACCEPT all -- anywhere anywhere PHYSDEV match
--physdev-out eth1 --physdev-is-bridged
Chain BF-cloudbr0-IN (1 references)
target prot opt source destination
v-16-VM all -- anywhere anywhere PHYSDEV match
--physdev-in vnet1 --physdev-is-bridged
s-17-VM all -- anywhere anywhere PHYSDEV match
--physdev-in vnet4 --physdev-is-bridged
Chain BF-cloudbr0-OUT (1 references)
target prot opt source destination
v-16-VM all -- anywhere anywhere PHYSDEV match
--physdev-out vnet1 --physdev-is-bridged
s-17-VM all -- anywhere anywhere PHYSDEV match
--physdev-out vnet4 --physdev-is-bridged
Chain r-3-VM (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match
--physdev-in vnet6 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
Chain s-17-VM (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match
--physdev-in vnet4 --physdev-is-bridged
Chain v-16-VM (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match
--physdev-in vnet1 --physdev-is-bridged
[root@fr-kvm1 network-scripts]#
Regards
On Thu, Aug 29, 2019 at 1:24 AM Andrija Panic <an...@gmail.com>
wrote:
> Can you do as I suggested please, vm on KVM vs VMs in DB - if zero VMs on
> KVM, then see if you can delete all VMs on ACS via GUI or DB.
>
> Then restart mgmt and enable the zone and give it some minutes and if
> problem persist, please share mgmt logs (please do so via pastebin or
> similar)...
>
> And answers my question please from previoa email.
>
> On Wed, Aug 28, 2019, 22:39 Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Any idea?
> >
> > On Wed, Aug 28, 2019 at 11:10 PM Fariborz Navidan <mdvlinquest@gmail.com
> >
> > wrote:
> >
> > > It is CentOS Linux release 7.6.1810 (Core) Agent is misleading. virsh
> > says
> > > no domain is in KVM.
> > >
> > > [root@fr-kvm1 ~]# virsh list --all
> > > Id Name State
> > > ----------------------------------------------------
> > >
> > > [root@fr-kvm1 ~]#
> > >
> > > On Wed, Aug 28, 2019 at 10:46 PM Andrija Panic <
> andrija.panic@gmail.com>
> > > wrote:
> > >
> > >> That is brave :) I mean running unsupported CentOS8 with ACS.
> Seriously,
> > >> if
> > >> not mistaken, we plan to support it in 4.14 only, and this might be
> > >> causing
> > >> issues as well.
> > >>
> > >> Your KVM host reports its already running 3 VMs, and that there are
> > >> pending
> > >> tasks on some of the VMs.
> > >>
> > >> Make sure that only VMs that exist in ACS database, also exist on KVM,
> > and
> > >> virsh destroy all others (perhaps disable zone, destroy any running
> > system
> > >> VMs and then make sure KVM vs DB vm list iz in correct sync.)
> > >>
> > >> Vlan 100 you were using for Public VLAN? Not sure how is your cloudbr1
> > >> (used for public traffic) configured and what vlans etc you are using
> > now.
> > >> How did you change vlan 100?
> > >>
> > >> Andrija
> > >>
> > >>
> > >> On Wed, Aug 28, 2019, 19:32 Fariborz Navidan <md...@gmail.com>
> > >> wrote:
> > >>
> > >> > Any idea? I have also already managed devpts issue which causes
> issues
> > >> on
> > >> > CentOS 8. I have copied devpts line from /etc/fstab from another
> host.
> > >> Also
> > >> > SELinux is already disabled
> > >> >
> > >> > On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <
> > mdvlinquest@gmail.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > This is agent log at debug log level:
> > >> > >
> > >> > > [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
> > >> > >
> > >> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
> > >> (Agent-Handler-2:null)
> > >> > > (logid:) Received response: Seq 1-1: { Ans: , MgmtId:
> > >> 157082893985413,
> > >> > > via: -1, Ver: v1, Flags: 100000,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
> > >> (Agent-Handler-2:null)
> > >> > > (logid:) Startup task cancelled
> > >> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
> > >> (Agent-Handler-2:null)
> > >> > > (logid:) Proccess agent startup answer, agent id = 0
> > >> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
> > >> (Agent-Handler-2:null)
> > >> > > (logid:) Set agent id 0
> > >> > > 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent]
> > >> (Agent-Handler-2:null)
> > >> > > (logid:) Adding a watch list
> > >> > > 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent]
> > >> (Agent-Handler-2:null)
> > >> > > (logid:) Startup Response Received: agent id = 0
> > >> > > 2019-08-28 18:59:16,277 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (UgentTask-1:null) (logid:) Executing:
> > >> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > >> > > get_rule_logs_for_vms
> > >> > > 2019-08-28 18:59:16,277 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (UgentTask-1:null) (logid:) Executing while with timeout : 1800000
> > >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100011,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.CleanupNetworkRulesCmd
> > >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
> > >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987393:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > >> > > 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
> > >> > > 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to
> > scan
> > >> 1
> > >> > > urls, producing 5 keys and 147 values
> > >> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> > >> > > java.lang.Comparable -> java.lang.Enum
> > >> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> > >> > > java.io.Serializable -> java.lang.Enum
> > >> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100111,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.CheckNetworkCommand
> > >> > > 2019-08-28 18:59:16,327 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (UgentTask-1:null) (logid:) Execution is successful.
> > >> > > 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (UgentTask-1:null) (logid:) Looking for libvirtd connection at:
> > >> > > qemu:///system
> > >> > > 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent]
> (UgentTask-1:null)
> > >> > > (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver:
> > v1,
> > >> > > Flags: 11,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,334 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing:
> > >> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > >> > > cleanup_rules
> > >> > > 2019-08-28 18:59:16,335 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831)
> > >> matchPifFileInDirectory:
> > >> > > file name 'eth1.100'
> > >> > > 2019-08-28 18:59:16,335 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing while with
> > timeout :
> > >> > > 1800000
> > >> > > 2019-08-28 18:59:16,335 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831)
> > >> matchPifFileInDirectory:
> > >> > > file name 'eth1'
> > >> > > 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987394:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,396 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
> > >> > > 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent]
> > >> (Agent-Handler-4:null)
> > >> > > (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: ,
> > >> MgmtId:
> > >> > > 157082893985413, via: 1, Ver: v1, Flags: 10,
> > >> > >
> > >>
> [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
> > >> > > 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent]
> > >> (Agent-Handler-1:null)
> > >> > > (logid:) Received response: Seq 0-2: { Ans: , MgmtId:
> > >> 157082893985413,
> > >> > > via: 1, Ver: v1, Flags: 100010,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100011,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.ModifyStoragePoolCommand
> > >> > > 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to
> create
> > >> > storage
> > >> > > pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in
> > >> libvirt
> > >> > > 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for
> libvirtd
> > >> > > connection at: qemu:///system
> > >> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing
> > defined
> > >> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
> > >> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch
> > storage
> > >> > pool
> > >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > >> > > 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for
> libvirtd
> > >> > > connection at: qemu:///system
> > >> > > 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully
> refreshed
> > >> pool
> > >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> > >> > 727711744
> > >> > > Available: 984168267776
> > >> > > 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987395:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100111,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100011,
> > >> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> > >> > }
> > >> > > 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.ModifySshKeysCommand
> > >> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.SetHostParamsCommand
> > >> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987397:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > >> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100011,
> > >> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> > >> > }
> > >> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.ModifySshKeysCommand
> > >> > > 2019-08-28 18:59:16,529 DEBUG
> > >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod
> 600
> > >> > > /root/.ssh/id_rsa.cloud
> > >> > > 2019-08-28 18:59:16,529 DEBUG
> > >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod
> 600
> > >> > > /root/.ssh/id_rsa.cloud
> > >> > > 2019-08-28 18:59:16,531 DEBUG
> > >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while
> with
> > >> > timeout
> > >> > > : 1800000
> > >> > > 2019-08-28 18:59:16,532 DEBUG
> > >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while
> with
> > >> > timeout
> > >> > > : 1800000
> > >> > > 2019-08-28 18:59:16,534 DEBUG
> > >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is
> > >> successful.
> > >> > > 2019-08-28 18:59:16,534 DEBUG
> > >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is
> > >> successful.
> > >> > > 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987398:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > >> > > 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987396:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > >> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100111,
> > >> > >
> > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}]
> > >> }
> > >> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.ReadyCommand
> > >> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent
> > ready
> > >> > > command, agent id = 1
> > >> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
> > >> > > 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is
> > >> processed
> > >> > > for agent id = 1
> > >> > > 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987399:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > >> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> > >> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> > >> > > 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1,
> > Ver:
> > >> v1,
> > >> > > Flags: 100111,
> > >> > >
> > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}]
> > >> }
> > >> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> > >> > > com.cloud.agent.api.ReadyCommand
> > >> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent
> > ready
> > >> > > command, agent id = 1
> > >> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
> > >> > > 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is
> > >> processed
> > >> > > for agent id = 1
> > >> > > 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
> > >> 1-4941856166108987400:
> > >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > >> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> > >> > > 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
> > >> > > com.cloud.agent.api.GetHostStatsCommand
> > >> > > 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
> > >> > > com.cloud.agent.api.GetStorageStatsCommand
> > >> > > 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch
> > storage
> > >> > pool
> > >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > >> > > 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for
> libvirtd
> > >> > > connection at: qemu:///system
> > >> > > 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to
> > >> refresh
> > >> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> > >> > > 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully
> refreshed
> > >> pool
> > >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> > >> > 727711744
> > >> > > Available: 984168267776
> > >> > > 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (Thread-8:null) (logid:) Looking for libvirtd connection at:
> > >> > qemu:///system
> > >> > > 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor]
> > >> (Thread-8:null)
> > >> > > (logid:) Found NFS storage pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88
> > >> in
> > >> > > libvirt, continuing
> > >> > > 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor]
> > >> (Thread-8:null)
> > >> > > (logid:) Executing:
> > >> > >
> > >> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> > -i
> > >> > > 192.168.0.2 -p /home/primary1 -m
> > >> > /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
> > >> > > -h 192.168.0.2
> > >> > > 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor]
> > >> (Thread-8:null)
> > >> > > (logid:) Executing while with timeout : 60000
> > >> > > 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor]
> > >> (Thread-8:null)
> > >> > > (logid:) Execution is successful.
> > >> > > 2019-08-28 19:00:16,278 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (UgentTask-2:null) (logid:) Executing:
> > >> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > >> > > get_rule_logs_for_vms
> > >> > > 2019-08-28 19:00:16,281 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (UgentTask-2:null) (logid:) Executing while with timeout : 1800000
> > >> > > 2019-08-28 19:00:16,331 DEBUG
> > [kvm.resource.LibvirtComputingResource]
> > >> > > (UgentTask-2:null) (logid:) Execution is successful.
> > >> > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (UgentTask-2:null) (logid:) Looking for libvirtd connection at:
> > >> > > qemu:///system
> > >> > > 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent]
> (UgentTask-2:null)
> > >> > > (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver:
> > v1,
> > >> > > Flags: 11,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent]
> > >> (Agent-Handler-4:null)
> > >> > > (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
> > >> > > 157082893985413, via: 1, Ver: v1, Flags: 100010,
> > >> > >
> > >> >
> > >>
> >
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
> > >> > > }
> > >> > > 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
> > >> > > com.cloud.agent.api.GetHostStatsCommand
> > >> > > 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
> > >> > > (agentRequest-Handler-3:null) (logid:79632973) Processing command:
> > >> > > com.cloud.agent.api.GetStorageStatsCommand
> > >> > > 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch
> > storage
> > >> > pool
> > >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > >> > > 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
> > >> > > (agentRequest-Handler-3:null) (logid:79632973) Looking for
> libvirtd
> > >> > > connection at: qemu:///system
> > >> > > 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to
> > >> refresh
> > >> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> > >> > > 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > >> > > (agentRequest-Handler-3:null) (logid:79632973) Succesfully
> refreshed
> > >> pool
> > >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> > >> > 727711744
> > >> > > Available: 984168267776
> > >> > > [root@fr-kvm1 primary1]#
> > >> > >
> > >> > > On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <
> > >> andrija.panic@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > >> I Don't see anything attached from agent.log? Make sure to put
> > agent
> > >> in
> > >> > >> debug mode before posting logs (google for "cloudstack agent
> debug
> > >> > mode",
> > >> > >> there is a simple sed command, restart agent afterwards).
> > >> > >>
> > >> > >> Andrija
> > >> > >>
> > >> > >> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <
> > mdvlinquest@gmail.com>
> > >> > >> wrote:
> > >> > >>
> > >> > >> > Hi,
> > >> > >> >
> > >> > >> > Also this is agent log
> > >> > >> >
> > >> > >> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <
> > >> > mdvlinquest@gmail.com
> > >> > >> >
> > >> > >> > wrote:
> > >> > >> >
> > >> > >> > > Any idea?
> > >> > >> > >
> > >> > >> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
> > >> > >> mdvlinquest@gmail.com>
> > >> > >> > > wrote:
> > >> > >> > >
> > >> > >> > >> Thanks. I changed VLAN ID and now they are kept starting for
> > >> along
> > >> > >> time.
> > >> > >> > >> Latest logs:
> > >> > >> > >>
> > >> > >> > >>
> > >> > >> > >> 2019-08-28 14:34:04,851 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:34:04,859 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has
> > been
> > >> > >> pending
> > >> > >> > for
> > >> > >> > >> 959 seconds
> > >> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has
> > been
> > >> > >> pending
> > >> > >> > for
> > >> > >> > >> 357 seconds
> > >> > >> > >> 2019-08-28 14:34:14,851 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:34:14,852 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found
> 0
> > >> > >> routers to
> > >> > >> > >> update status.
> > >> > >> > >> 2019-08-28 14:34:14,854 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found
> 0
> > >> VPC
> > >> > >> > networks
> > >> > >> > >> to update Redundant State.
> > >> > >> > >> 2019-08-28 14:34:14,857 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found
> 0
> > >> > >> networks
> > >> > >> > to
> > >> > >> > >> update RvR status.
> > >> > >> > >> 2019-08-28 14:34:14,859 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598229
> > >> > >> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b)
> (logid:69ceb90d)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b)
> (logid:69ceb90d)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598229
> > >> > >> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > >> > Resetting
> > >> > >> > hosts
> > >> > >> > >> suitable for reconnect
> > >> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > >> > Completed
> > >> > >> > >> resetting hosts suitable for reconnect
> > >> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > >> > Acquiring
> > >> > >> > hosts
> > >> > >> > >> for clusters already owned by this management server
> > >> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > >> > Completed
> > >> > >> > >> acquiring hosts for clusters already owned by this
> management
> > >> > server
> > >> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > >> > Acquiring
> > >> > >> > hosts
> > >> > >> > >> for clusters not owned by any management server
> > >> > >> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > >> > Completed
> > >> > >> > >> acquiring hosts for clusters not owned by any management
> > server
> > >> > >> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> > >> > >> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565)
> (logid:f8b01d6e)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565)
> (logid:f8b01d6e)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> > >> > >> > >> 2019-08-28 14:34:24,857 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:34:24,865 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
> > >> > >> fr-kvm1.r9host.com)
> > >> > >> > >> 2019-08-28 14:34:24,974 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Process host VM
> state
> > >> > report
> > >> > >> > from
> > >> > >> > >> ping process. host: 1
> > >> > >> > >> 2019-08-28 14:34:24,975 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Process VM state
> > report.
> > >> > >> host:
> > >> > >> > 1,
> > >> > >> > >> number of records in report: 1
> > >> > >> > >> 2019-08-28 14:34:24,975 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report.
> host:
> > >> 1,
> > >> > vm
> > >> > >> id:
> > >> > >> > >> 103, power state: PowerOn
> > >> > >> > >> 2019-08-28 14:34:24,976 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> > >> updated.
> > >> > >> > host:
> > >> > >> > >> 1, vm id: 103, power state: PowerOn
> > >> > >> > >> 2019-08-28 14:34:24,977 INFO
> > [c.c.v.VirtualMachineManagerImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job
> > or
> > >> HA
> > >> > >> tasks
> > >> > >> > >> working on the VM. vm id: 103, postpone power-change report
> by
> > >> > >> resetting
> > >> > >> > >> power-change counters
> > >> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM
> report.
> > >> > >> current
> > >> > >> > >> time: 1566995664980
> > >> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
> > >> host:
> > >> > >> 1, vm
> > >> > >> > >> id: 101, power state: PowerReportMissing, last state update:
> > >> > >> > 1566995544000
> > >> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time
> > since
> > >> > last
> > >> > >> > state
> > >> > >> > >> update(120980ms) has passed graceful period
> > >> > >> > >> 2019-08-28 14:34:24,982 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> > >> updated.
> > >> > >> > host:
> > >> > >> > >> 1, vm id: 101, power state: PowerReportMissing
> > >> > >> > >> 2019-08-28 14:34:24,982 INFO
> > [c.c.v.VirtualMachineManagerImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job
> > or
> > >> HA
> > >> > >> tasks
> > >> > >> > >> working on the VM. vm id: 101, postpone power-change report
> by
> > >> > >> resetting
> > >> > >> > >> power-change counters
> > >> > >> > >> 2019-08-28 14:34:24,984 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
> > >> host:
> > >> > >> 1, vm
> > >> > >> > >> id: 102, power state: PowerReportMissing, last state update:
> > >> > >> > 1566995544000
> > >> > >> > >> 2019-08-28 14:34:24,984 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time
> > since
> > >> > last
> > >> > >> > state
> > >> > >> > >> update(120980ms) has passed graceful period
> > >> > >> > >> 2019-08-28 14:34:24,986 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> > >> updated.
> > >> > >> > host:
> > >> > >> > >> 1, vm id: 102, power state: PowerReportMissing
> > >> > >> > >> 2019-08-28 14:34:24,986 INFO
> > [c.c.v.VirtualMachineManagerImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job
> > or
> > >> HA
> > >> > >> tasks
> > >> > >> > >> working on the VM. vm id: 102, postpone power-change report
> by
> > >> > >> resetting
> > >> > >> > >> power-change counters
> > >> > >> > >> 2019-08-28 14:34:24,987 DEBUG
> > >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> > >> (AgentManager-Handler-15:null) (logid:) Done with process of
> > VM
> > >> > state
> > >> > >> > >> report. host: 1
> > >> > >> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598231
> > >> > >> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176)
> (logid:d2576b2d)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176)
> (logid:d2576b2d)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598231
> > >> > >> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> > >> > >> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397)
> (logid:6a21df96)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397)
> (logid:6a21df96)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> > >> > >> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> > >> > >> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling
> > >> > Monitor
> > >> > >> is
> > >> > >> > >> running...
> > >> > >> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> > >> > >> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f)
> (logid:17ce85f1)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f)
> (logid:17ce85f1)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> > >> > >> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> > >> > >> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df)
> (logid:579437b8)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df)
> (logid:579437b8)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> > >> > >> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> > >> > >> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4)
> (logid:1929ff2c)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4)
> (logid:1929ff2c)
> > >> Api
> > >> > >> > >> Discovery plugin was unable to find an api by that name or
> > >> process
> > >> > >> any
> > >> > >> > apis
> > >> > >> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4)
> (logid:1929ff2c)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> > >> > >> > >> 2019-08-28 14:34:34,851 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:34:34,861 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> > >> > >> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315)
> (logid:12e0b85c)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315)
> (logid:12e0b85c)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> > >> > >> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> > >> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07)
> > >> HostStatsCollector
> > >> > >> is
> > >> > >> > >> running...
> > >> > >> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> > >> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
> > >> > >> > 1-5778962746846479041:
> > >> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> > >> fr-kvm1.r9host.com
> > >> > >> > ),
> > >> > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> > >> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> > >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> > >> StorageCollector
> > >> > is
> > >> > >> > >> running...
> > >> > >> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> > >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no
> > >> > >> secondary
> > >> > >> > >> storage VM for secondary storage host Secondary1
> > >> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> > >> > >> > getCommandHostDelegation:
> > >> > >> > >> class com.cloud.agent.api.GetStorageStatsCommand
> > >> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> > >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are
> > >> returning
> > >> > the
> > >> > >> > >> default host to execute commands because the command is not
> of
> > >> Copy
> > >> > >> > type.
> > >> > >> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> > >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
> > >> > >> > 1-5778962746846479042:
> > >> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> > >> fr-kvm1.r9host.com
> > >> > >> > ),
> > >> > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >> > >> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> > >> > >> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339)
> (logid:fd475e73)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339)
> (logid:fd475e73)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> > >> > >> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> > >> > >> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774)
> (logid:53245a8d)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774)
> (logid:53245a8d)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> > >> > >> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> > >> > >> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26)
> (logid:a29f4b7b)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26)
> (logid:a29f4b7b)
> > >> Api
> > >> > >> > >> Discovery plugin was unable to find an api by that name or
> > >> process
> > >> > >> any
> > >> > >> > apis
> > >> > >> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26)
> (logid:a29f4b7b)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> > >> > >> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598240
> > >> > >> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7)
> (logid:087bc8bb)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7)
> (logid:087bc8bb)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598240
> > >> > >> > >> 2019-08-28 14:34:44,850 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:34:44,852 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found
> 0
> > >> > >> routers to
> > >> > >> > >> update status.
> > >> > >> > >> 2019-08-28 14:34:44,855 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found
> 0
> > >> VPC
> > >> > >> > networks
> > >> > >> > >> to update Redundant State.
> > >> > >> > >> 2019-08-28 14:34:44,857 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found
> 0
> > >> > >> networks
> > >> > >> > to
> > >> > >> > >> update RvR status.
> > >> > >> > >> 2019-08-28 14:34:44,859 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> > >> > >> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d)
> (logid:6e637323)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:48,215 WARN
> > >> > [c.c.a.d.ParamGenericValidationWorker]
> > >> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d)
> (logid:6e637323)
> > >> > >> Received
> > >> > >> > >> unknown parameters for command listSystemVms. Unknown
> > >> parameters :
> > >> > >> > listall
> > >> > >> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d)
> (logid:6e637323)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> > >> > >> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
> > >> > >> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73)
> (logid:1065f9c5)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73)
> (logid:1065f9c5)
> > >> > >> > >>>Searching
> > >> > >> > >> for hosts>>>
> > >> > >> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73)
> (logid:1065f9c5)
> > >> > >> > >> >>>Generating Response>>>
> > >> > >> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73)
> (logid:1065f9c5)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
> > >> > >> > >> 2019-08-28 14:34:54,843 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:34:54,845 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> > >> > >> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c)
> (logid:3489b330)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c)
> (logid:3489b330)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> > >> > >> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> > >> > >> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde)
> (logid:2b550fd0)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde)
> (logid:2b550fd0)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> > >> > >> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598245
> > >> > >> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882)
> (logid:50c19de5)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882)
> (logid:50c19de5)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598245
> > >> > >> > >> 2019-08-28 14:35:04,849 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:35:04,851 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> > >> > >> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150)
> (logid:2682bbeb)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150)
> (logid:2682bbeb)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> > >> > >> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> > >> > >> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42)
> (logid:a415b9a3)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42)
> (logid:a415b9a3)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> > >> > >> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> > >> > >> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704)
> (logid:00800f6e)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704)
> (logid:00800f6e)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> > >> > >> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> > >> > >> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5)
> (logid:eca6e20c)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5)
> (logid:eca6e20c)
> > >> Api
> > >> > >> > >> Discovery plugin was unable to find an api by that name or
> > >> process
> > >> > >> any
> > >> > >> > apis
> > >> > >> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5)
> (logid:eca6e20c)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> > >> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has
> > been
> > >> > >> pending
> > >> > >> > for
> > >> > >> > >> 1019 seconds
> > >> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has
> > been
> > >> > >> pending
> > >> > >> > for
> > >> > >> > >> 417 seconds
> > >> > >> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> > >> > >> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4)
> (logid:185d38c3)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4)
> (logid:185d38c3)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> > >> > >> > >> 2019-08-28 14:35:14,844 INFO
> [c.c.r.ResourceLimitManagerImpl]
> > >> > >> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195)
> Started
> > >> > >> resource
> > >> > >> > >> counters recalculation periodic task.
> > >> > >> > >> 2019-08-28 14:35:14,848 DEBUG
> > >> > [c.c.n.ExternalDeviceUsageManagerImpl]
> > >> > >> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32)
> > >> External
> > >> > >> > devices
> > >> > >> > >> are not used. Skipping external device usage collection
> > >> > >> > >> 2019-08-28 14:35:14,851 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:35:14,852 DEBUG
> [c.c.s.s.SnapshotSchedulerImpl]
> > >> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
> > >> > >> > scheduler.poll is
> > >> > >> > >> being called at 2019-08-28 12:35:14 GMT
> > >> > >> > >> 2019-08-28 14:35:14,857 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End
> > >> cleanup
> > >> > >> > expired
> > >> > >> > >> async-jobs
> > >> > >> > >> 2019-08-28 14:35:14,857 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found
> 0
> > >> > >> routers to
> > >> > >> > >> update status.
> > >> > >> > >> 2019-08-28 14:35:14,857 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > (RouterMonitor-1:ctx-7cb475c8)
> > >> > >> > >> (logid:cc3e421d) Found 0 running routers.
> > >> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found
> 0
> > >> VPC
> > >> > >> > networks
> > >> > >> > >> to update Redundant State.
> > >> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> [c.c.s.s.SnapshotSchedulerImpl]
> > >> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0
> > >> snapshots to
> > >> > >> be
> > >> > >> > >> executed at 2019-08-28 12:35:14 GMT
> > >> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> > >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found
> 0
> > >> > >> networks
> > >> > >> > to
> > >> > >> > >> update RvR status.
> > >> > >> > >> 2019-08-28 14:35:14,906 INFO
> [o.a.c.e.o.NetworkOrchestrator]
> > >> > >> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
> > >> > >> > NetworkGarbageCollector
> > >> > >> > >> uses '600' seconds for GC interval.
> > >> > >> > >> 2019-08-28 14:35:14,925 INFO
> > >> [c.c.h.HighAvailabilityManagerImpl]
> > >> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of
> usage
> > >> > server
> > >> > >> > >> 2019-08-28 14:35:14,928 DEBUG
> > >> [c.c.h.HighAvailabilityManagerImpl]
> > >> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running?
> > true,
> > >> > >> > heartbeat:
> > >> > >> > >> Wed Aug 28 14:35:12 CEST 2019
> > >> > >> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> > >> > >> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468)
> (logid:d2ff137a)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468)
> (logid:d2ff137a)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> > >> > >> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> > >> > >> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030)
> (logid:8c0fb627)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030)
> (logid:8c0fb627)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> > >> > >> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> > >> > >> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37)
> (logid:b31fa0e8)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37)
> (logid:b31fa0e8)
> > >> Api
> > >> > >> > >> Discovery plugin was unable to find an api by that name or
> > >> process
> > >> > >> any
> > >> > >> > apis
> > >> > >> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37)
> (logid:b31fa0e8)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >>
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> > >> > >> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6)
> ===START===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598254
> > >> > >> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> > >> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f)
> (logid:c0fd2ac6)
> > >> > CIDRs
> > >> > >> > from
> > >> > >> > >> which account
> > >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >> > >> > allowed
> > >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> > >> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f)
> (logid:c0fd2ac6)
> > >> > >> ===END===
> > >> > >> > >> 137.74.35.102 -- GET
> > >> > >> > >> command=listInfrastructure&response=json&_=1566995598254
> > >> > >> > >> 2019-08-28 14:35:24,852 INFO
> > [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f)
> Begin
> > >> > cleanup
> > >> > >> > >> expired async-jobs
> > >> > >> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.
>
Re: Unable to start System VMs
Posted by Andrija Panic <an...@gmail.com>.
Can you do as I suggested please, vm on KVM vs VMs in DB - if zero VMs on
KVM, then see if you can delete all VMs on ACS via GUI or DB.
Then restart mgmt and enable the zone and give it some minutes and if
problem persist, please share mgmt logs (please do so via pastebin or
similar)...
And answers my question please from previoa email.
On Wed, Aug 28, 2019, 22:39 Fariborz Navidan <md...@gmail.com> wrote:
> Any idea?
>
> On Wed, Aug 28, 2019 at 11:10 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > It is CentOS Linux release 7.6.1810 (Core) Agent is misleading. virsh
> says
> > no domain is in KVM.
> >
> > [root@fr-kvm1 ~]# virsh list --all
> > Id Name State
> > ----------------------------------------------------
> >
> > [root@fr-kvm1 ~]#
> >
> > On Wed, Aug 28, 2019 at 10:46 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> >> That is brave :) I mean running unsupported CentOS8 with ACS. Seriously,
> >> if
> >> not mistaken, we plan to support it in 4.14 only, and this might be
> >> causing
> >> issues as well.
> >>
> >> Your KVM host reports its already running 3 VMs, and that there are
> >> pending
> >> tasks on some of the VMs.
> >>
> >> Make sure that only VMs that exist in ACS database, also exist on KVM,
> and
> >> virsh destroy all others (perhaps disable zone, destroy any running
> system
> >> VMs and then make sure KVM vs DB vm list iz in correct sync.)
> >>
> >> Vlan 100 you were using for Public VLAN? Not sure how is your cloudbr1
> >> (used for public traffic) configured and what vlans etc you are using
> now.
> >> How did you change vlan 100?
> >>
> >> Andrija
> >>
> >>
> >> On Wed, Aug 28, 2019, 19:32 Fariborz Navidan <md...@gmail.com>
> >> wrote:
> >>
> >> > Any idea? I have also already managed devpts issue which causes issues
> >> on
> >> > CentOS 8. I have copied devpts line from /etc/fstab from another host.
> >> Also
> >> > SELinux is already disabled
> >> >
> >> > On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <
> mdvlinquest@gmail.com
> >> >
> >> > wrote:
> >> >
> >> > > This is agent log at debug log level:
> >> > >
> >> > > [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
> >> > >
> >> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
> >> (Agent-Handler-2:null)
> >> > > (logid:) Received response: Seq 1-1: { Ans: , MgmtId:
> >> 157082893985413,
> >> > > via: -1, Ver: v1, Flags: 100000,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
> >> (Agent-Handler-2:null)
> >> > > (logid:) Startup task cancelled
> >> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
> >> (Agent-Handler-2:null)
> >> > > (logid:) Proccess agent startup answer, agent id = 0
> >> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
> >> (Agent-Handler-2:null)
> >> > > (logid:) Set agent id 0
> >> > > 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent]
> >> (Agent-Handler-2:null)
> >> > > (logid:) Adding a watch list
> >> > > 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent]
> >> (Agent-Handler-2:null)
> >> > > (logid:) Startup Response Received: agent id = 0
> >> > > 2019-08-28 18:59:16,277 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (UgentTask-1:null) (logid:) Executing:
> >> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> >> > > get_rule_logs_for_vms
> >> > > 2019-08-28 18:59:16,277 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (UgentTask-1:null) (logid:) Executing while with timeout : 1800000
> >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100011,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.CleanupNetworkRulesCmd
> >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
> >> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987393:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> >> > > 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
> >> > >
> >> > >
> >> >
> >>
> jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
> >> > > 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to
> scan
> >> 1
> >> > > urls, producing 5 keys and 147 values
> >> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> >> > > java.lang.Comparable -> java.lang.Enum
> >> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> >> > > java.io.Serializable -> java.lang.Enum
> >> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100111,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.CheckNetworkCommand
> >> > > 2019-08-28 18:59:16,327 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (UgentTask-1:null) (logid:) Execution is successful.
> >> > > 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (UgentTask-1:null) (logid:) Looking for libvirtd connection at:
> >> > > qemu:///system
> >> > > 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null)
> >> > > (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver:
> v1,
> >> > > Flags: 11,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,334 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing:
> >> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> >> > > cleanup_rules
> >> > > 2019-08-28 18:59:16,335 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831)
> >> matchPifFileInDirectory:
> >> > > file name 'eth1.100'
> >> > > 2019-08-28 18:59:16,335 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing while with
> timeout :
> >> > > 1800000
> >> > > 2019-08-28 18:59:16,335 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831)
> >> matchPifFileInDirectory:
> >> > > file name 'eth1'
> >> > > 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987394:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,396 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
> >> > > 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent]
> >> (Agent-Handler-4:null)
> >> > > (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: ,
> >> MgmtId:
> >> > > 157082893985413, via: 1, Ver: v1, Flags: 10,
> >> > >
> >> [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
> >> > > 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent]
> >> (Agent-Handler-1:null)
> >> > > (logid:) Received response: Seq 0-2: { Ans: , MgmtId:
> >> 157082893985413,
> >> > > via: 1, Ver: v1, Flags: 100010,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100011,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.ModifyStoragePoolCommand
> >> > > 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create
> >> > storage
> >> > > pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in
> >> libvirt
> >> > > 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> >> > > connection at: qemu:///system
> >> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing
> defined
> >> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
> >> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch
> storage
> >> > pool
> >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> >> > > 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> >> > > connection at: qemu:///system
> >> > > 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed
> >> pool
> >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> >> > 727711744
> >> > > Available: 984168267776
> >> > > 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987395:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100111,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
> >> > > }
> >> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100011,
> >> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> >> > }
> >> > > 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.ModifySshKeysCommand
> >> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.SetHostParamsCommand
> >> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987397:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> >> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100011,
> >> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> >> > }
> >> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.ModifySshKeysCommand
> >> > > 2019-08-28 18:59:16,529 DEBUG
> >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600
> >> > > /root/.ssh/id_rsa.cloud
> >> > > 2019-08-28 18:59:16,529 DEBUG
> >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600
> >> > > /root/.ssh/id_rsa.cloud
> >> > > 2019-08-28 18:59:16,531 DEBUG
> >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with
> >> > timeout
> >> > > : 1800000
> >> > > 2019-08-28 18:59:16,532 DEBUG
> >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with
> >> > timeout
> >> > > : 1800000
> >> > > 2019-08-28 18:59:16,534 DEBUG
> >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is
> >> successful.
> >> > > 2019-08-28 18:59:16,534 DEBUG
> >> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is
> >> successful.
> >> > > 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987398:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> >> > > 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987396:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> >> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> >> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100111,
> >> > >
> [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}]
> >> }
> >> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.ReadyCommand
> >> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent
> ready
> >> > > command, agent id = 1
> >> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
> >> > > 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is
> >> processed
> >> > > for agent id = 1
> >> > > 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987399:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> >> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> >> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> >> > > 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1,
> Ver:
> >> v1,
> >> > > Flags: 100111,
> >> > >
> [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}]
> >> }
> >> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> >> > > com.cloud.agent.api.ReadyCommand
> >> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent
> ready
> >> > > command, agent id = 1
> >> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
> >> > > 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is
> >> processed
> >> > > for agent id = 1
> >> > > 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
> >> 1-4941856166108987400:
> >> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> >> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> >> > > 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
> >> > > com.cloud.agent.api.GetHostStatsCommand
> >> > > 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
> >> > > com.cloud.agent.api.GetStorageStatsCommand
> >> > > 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch
> storage
> >> > pool
> >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> >> > > 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd
> >> > > connection at: qemu:///system
> >> > > 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to
> >> refresh
> >> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> >> > > 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed
> >> pool
> >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> >> > 727711744
> >> > > Available: 984168267776
> >> > > 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (Thread-8:null) (logid:) Looking for libvirtd connection at:
> >> > qemu:///system
> >> > > 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor]
> >> (Thread-8:null)
> >> > > (logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> >> in
> >> > > libvirt, continuing
> >> > > 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor]
> >> (Thread-8:null)
> >> > > (logid:) Executing:
> >> > >
> >> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> -i
> >> > > 192.168.0.2 -p /home/primary1 -m
> >> > /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
> >> > > -h 192.168.0.2
> >> > > 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor]
> >> (Thread-8:null)
> >> > > (logid:) Executing while with timeout : 60000
> >> > > 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor]
> >> (Thread-8:null)
> >> > > (logid:) Execution is successful.
> >> > > 2019-08-28 19:00:16,278 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (UgentTask-2:null) (logid:) Executing:
> >> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> >> > > get_rule_logs_for_vms
> >> > > 2019-08-28 19:00:16,281 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (UgentTask-2:null) (logid:) Executing while with timeout : 1800000
> >> > > 2019-08-28 19:00:16,331 DEBUG
> [kvm.resource.LibvirtComputingResource]
> >> > > (UgentTask-2:null) (logid:) Execution is successful.
> >> > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (UgentTask-2:null) (logid:) Looking for libvirtd connection at:
> >> > > qemu:///system
> >> > > 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null)
> >> > > (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver:
> v1,
> >> > > Flags: 11,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent]
> >> (Agent-Handler-4:null)
> >> > > (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
> >> > > 157082893985413, via: 1, Ver: v1, Flags: 100010,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
> >> > > }
> >> > > 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
> >> > > com.cloud.agent.api.GetHostStatsCommand
> >> > > 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
> >> > > (agentRequest-Handler-3:null) (logid:79632973) Processing command:
> >> > > com.cloud.agent.api.GetStorageStatsCommand
> >> > > 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch
> storage
> >> > pool
> >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> >> > > 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
> >> > > (agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd
> >> > > connection at: qemu:///system
> >> > > 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to
> >> refresh
> >> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> >> > > 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> >> > > (agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed
> >> pool
> >> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> >> > 727711744
> >> > > Available: 984168267776
> >> > > [root@fr-kvm1 primary1]#
> >> > >
> >> > > On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <
> >> andrija.panic@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> I Don't see anything attached from agent.log? Make sure to put
> agent
> >> in
> >> > >> debug mode before posting logs (google for "cloudstack agent debug
> >> > mode",
> >> > >> there is a simple sed command, restart agent afterwards).
> >> > >>
> >> > >> Andrija
> >> > >>
> >> > >> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <
> mdvlinquest@gmail.com>
> >> > >> wrote:
> >> > >>
> >> > >> > Hi,
> >> > >> >
> >> > >> > Also this is agent log
> >> > >> >
> >> > >> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <
> >> > mdvlinquest@gmail.com
> >> > >> >
> >> > >> > wrote:
> >> > >> >
> >> > >> > > Any idea?
> >> > >> > >
> >> > >> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
> >> > >> mdvlinquest@gmail.com>
> >> > >> > > wrote:
> >> > >> > >
> >> > >> > >> Thanks. I changed VLAN ID and now they are kept starting for
> >> along
> >> > >> time.
> >> > >> > >> Latest logs:
> >> > >> > >>
> >> > >> > >>
> >> > >> > >> 2019-08-28 14:34:04,851 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:34:04,859 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has
> been
> >> > >> pending
> >> > >> > for
> >> > >> > >> 959 seconds
> >> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has
> been
> >> > >> pending
> >> > >> > for
> >> > >> > >> 357 seconds
> >> > >> > >> 2019-08-28 14:34:14,851 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:34:14,852 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> >> > >> routers to
> >> > >> > >> update status.
> >> > >> > >> 2019-08-28 14:34:14,854 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> >> VPC
> >> > >> > networks
> >> > >> > >> to update Redundant State.
> >> > >> > >> 2019-08-28 14:34:14,857 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> >> > >> networks
> >> > >> > to
> >> > >> > >> update RvR status.
> >> > >> > >> 2019-08-28 14:34:14,859 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598229
> >> > >> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598229
> >> > >> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> >> > Resetting
> >> > >> > hosts
> >> > >> > >> suitable for reconnect
> >> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> >> > Completed
> >> > >> > >> resetting hosts suitable for reconnect
> >> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> >> > Acquiring
> >> > >> > hosts
> >> > >> > >> for clusters already owned by this management server
> >> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> >> > Completed
> >> > >> > >> acquiring hosts for clusters already owned by this management
> >> > server
> >> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> >> > Acquiring
> >> > >> > hosts
> >> > >> > >> for clusters not owned by any management server
> >> > >> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> >> > Completed
> >> > >> > >> acquiring hosts for clusters not owned by any management
> server
> >> > >> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> >> > >> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> >> > >> > >> 2019-08-28 14:34:24,857 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:34:24,865 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
> >> > >> fr-kvm1.r9host.com)
> >> > >> > >> 2019-08-28 14:34:24,974 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state
> >> > report
> >> > >> > from
> >> > >> > >> ping process. host: 1
> >> > >> > >> 2019-08-28 14:34:24,975 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Process VM state
> report.
> >> > >> host:
> >> > >> > 1,
> >> > >> > >> number of records in report: 1
> >> > >> > >> 2019-08-28 14:34:24,975 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host:
> >> 1,
> >> > vm
> >> > >> id:
> >> > >> > >> 103, power state: PowerOn
> >> > >> > >> 2019-08-28 14:34:24,976 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> >> updated.
> >> > >> > host:
> >> > >> > >> 1, vm id: 103, power state: PowerOn
> >> > >> > >> 2019-08-28 14:34:24,977 INFO
> [c.c.v.VirtualMachineManagerImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job
> or
> >> HA
> >> > >> tasks
> >> > >> > >> working on the VM. vm id: 103, postpone power-change report by
> >> > >> resetting
> >> > >> > >> power-change counters
> >> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report.
> >> > >> current
> >> > >> > >> time: 1566995664980
> >> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
> >> host:
> >> > >> 1, vm
> >> > >> > >> id: 101, power state: PowerReportMissing, last state update:
> >> > >> > 1566995544000
> >> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time
> since
> >> > last
> >> > >> > state
> >> > >> > >> update(120980ms) has passed graceful period
> >> > >> > >> 2019-08-28 14:34:24,982 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> >> updated.
> >> > >> > host:
> >> > >> > >> 1, vm id: 101, power state: PowerReportMissing
> >> > >> > >> 2019-08-28 14:34:24,982 INFO
> [c.c.v.VirtualMachineManagerImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job
> or
> >> HA
> >> > >> tasks
> >> > >> > >> working on the VM. vm id: 101, postpone power-change report by
> >> > >> resetting
> >> > >> > >> power-change counters
> >> > >> > >> 2019-08-28 14:34:24,984 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
> >> host:
> >> > >> 1, vm
> >> > >> > >> id: 102, power state: PowerReportMissing, last state update:
> >> > >> > 1566995544000
> >> > >> > >> 2019-08-28 14:34:24,984 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time
> since
> >> > last
> >> > >> > state
> >> > >> > >> update(120980ms) has passed graceful period
> >> > >> > >> 2019-08-28 14:34:24,986 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> >> updated.
> >> > >> > host:
> >> > >> > >> 1, vm id: 102, power state: PowerReportMissing
> >> > >> > >> 2019-08-28 14:34:24,986 INFO
> [c.c.v.VirtualMachineManagerImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job
> or
> >> HA
> >> > >> tasks
> >> > >> > >> working on the VM. vm id: 102, postpone power-change report by
> >> > >> resetting
> >> > >> > >> power-change counters
> >> > >> > >> 2019-08-28 14:34:24,987 DEBUG
> >> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> > >> (AgentManager-Handler-15:null) (logid:) Done with process of
> VM
> >> > state
> >> > >> > >> report. host: 1
> >> > >> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598231
> >> > >> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598231
> >> > >> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> >> > >> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> >> > >> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> >> > >> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling
> >> > Monitor
> >> > >> is
> >> > >> > >> running...
> >> > >> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> >> > >> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> >> > >> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> >> > >> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> >> > >> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> >> > >> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> >> Api
> >> > >> > >> Discovery plugin was unable to find an api by that name or
> >> process
> >> > >> any
> >> > >> > apis
> >> > >> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> >> > >> > >> 2019-08-28 14:34:34,851 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:34:34,861 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> >> > >> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> >> > >> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> >> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07)
> >> HostStatsCollector
> >> > >> is
> >> > >> > >> running...
> >> > >> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> >> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
> >> > >> > 1-5778962746846479041:
> >> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> > >> fr-kvm1.r9host.com
> >> > >> > ),
> >> > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> >> StorageCollector
> >> > is
> >> > >> > >> running...
> >> > >> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no
> >> > >> secondary
> >> > >> > >> storage VM for secondary storage host Secondary1
> >> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> >> > >> > getCommandHostDelegation:
> >> > >> > >> class com.cloud.agent.api.GetStorageStatsCommand
> >> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are
> >> returning
> >> > the
> >> > >> > >> default host to execute commands because the command is not of
> >> Copy
> >> > >> > type.
> >> > >> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> >> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
> >> > >> > 1-5778962746846479042:
> >> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> > >> fr-kvm1.r9host.com
> >> > >> > ),
> >> > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >> > >> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> >> > >> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> >> > >> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> >> > >> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> >> > >> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> >> > >> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> >> Api
> >> > >> > >> Discovery plugin was unable to find an api by that name or
> >> process
> >> > >> any
> >> > >> > apis
> >> > >> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> >> > >> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598240
> >> > >> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598240
> >> > >> > >> 2019-08-28 14:34:44,850 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:34:44,852 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> >> > >> routers to
> >> > >> > >> update status.
> >> > >> > >> 2019-08-28 14:34:44,855 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> >> VPC
> >> > >> > networks
> >> > >> > >> to update Redundant State.
> >> > >> > >> 2019-08-28 14:34:44,857 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> >> > >> networks
> >> > >> > to
> >> > >> > >> update RvR status.
> >> > >> > >> 2019-08-28 14:34:44,859 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> >> > >> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:48,215 WARN
> >> > [c.c.a.d.ParamGenericValidationWorker]
> >> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> >> > >> Received
> >> > >> > >> unknown parameters for command listSystemVms. Unknown
> >> parameters :
> >> > >> > listall
> >> > >> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> >> > >> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
> >> > >> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> > >> > >>>Searching
> >> > >> > >> for hosts>>>
> >> > >> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> > >> > >> >>>Generating Response>>>
> >> > >> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
> >> > >> > >> 2019-08-28 14:34:54,843 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:34:54,845 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> >> > >> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> >> > >> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> >> > >> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> >> > >> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598245
> >> > >> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598245
> >> > >> > >> 2019-08-28 14:35:04,849 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:35:04,851 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> >> > >> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> >> > >> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> >> > >> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> >> > >> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> >> > >> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> >> > >> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> >> > >> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> >> Api
> >> > >> > >> Discovery plugin was unable to find an api by that name or
> >> process
> >> > >> any
> >> > >> > apis
> >> > >> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> >> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has
> been
> >> > >> pending
> >> > >> > for
> >> > >> > >> 1019 seconds
> >> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has
> been
> >> > >> pending
> >> > >> > for
> >> > >> > >> 417 seconds
> >> > >> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> >> > >> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> >> > >> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
> >> > >> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started
> >> > >> resource
> >> > >> > >> counters recalculation periodic task.
> >> > >> > >> 2019-08-28 14:35:14,848 DEBUG
> >> > [c.c.n.ExternalDeviceUsageManagerImpl]
> >> > >> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32)
> >> External
> >> > >> > devices
> >> > >> > >> are not used. Skipping external device usage collection
> >> > >> > >> 2019-08-28 14:35:14,851 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
> >> > >> > scheduler.poll is
> >> > >> > >> being called at 2019-08-28 12:35:14 GMT
> >> > >> > >> 2019-08-28 14:35:14,857 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End
> >> cleanup
> >> > >> > expired
> >> > >> > >> async-jobs
> >> > >> > >> 2019-08-28 14:35:14,857 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> >> > >> routers to
> >> > >> > >> update status.
> >> > >> > >> 2019-08-28 14:35:14,857 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > (RouterMonitor-1:ctx-7cb475c8)
> >> > >> > >> (logid:cc3e421d) Found 0 running routers.
> >> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> >> VPC
> >> > >> > networks
> >> > >> > >> to update Redundant State.
> >> > >> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0
> >> snapshots to
> >> > >> be
> >> > >> > >> executed at 2019-08-28 12:35:14 GMT
> >> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> >> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> >> > >> networks
> >> > >> > to
> >> > >> > >> update RvR status.
> >> > >> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
> >> > >> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
> >> > >> > NetworkGarbageCollector
> >> > >> > >> uses '600' seconds for GC interval.
> >> > >> > >> 2019-08-28 14:35:14,925 INFO
> >> [c.c.h.HighAvailabilityManagerImpl]
> >> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage
> >> > server
> >> > >> > >> 2019-08-28 14:35:14,928 DEBUG
> >> [c.c.h.HighAvailabilityManagerImpl]
> >> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running?
> true,
> >> > >> > heartbeat:
> >> > >> > >> Wed Aug 28 14:35:12 CEST 2019
> >> > >> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> >> > >> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> >> > >> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> >> > >> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> >> > >> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> >> > >> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> >> Api
> >> > >> > >> Discovery plugin was unable to find an api by that name or
> >> process
> >> > >> any
> >> > >> > apis
> >> > >> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >>
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> >> > >> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598254
> >> > >> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> >> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> >> > CIDRs
> >> > >> > from
> >> > >> > >> which account
> >> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > >> > allowed
> >> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> >> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> >> > >> ===END===
> >> > >> > >> 137.74.35.102 -- GET
> >> > >> > >> command=listInfrastructure&response=json&_=1566995598254
> >> > >> > >> 2019-08-28 14:35:24,852 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin
> >> > cleanup
> >> > >> > >> expired async-jobs
> >> > >> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
Any idea?
On Wed, Aug 28, 2019 at 11:10 PM Fariborz Navidan <md...@gmail.com>
wrote:
> It is CentOS Linux release 7.6.1810 (Core) Agent is misleading. virsh says
> no domain is in KVM.
>
> [root@fr-kvm1 ~]# virsh list --all
> Id Name State
> ----------------------------------------------------
>
> [root@fr-kvm1 ~]#
>
> On Wed, Aug 28, 2019 at 10:46 PM Andrija Panic <an...@gmail.com>
> wrote:
>
>> That is brave :) I mean running unsupported CentOS8 with ACS. Seriously,
>> if
>> not mistaken, we plan to support it in 4.14 only, and this might be
>> causing
>> issues as well.
>>
>> Your KVM host reports its already running 3 VMs, and that there are
>> pending
>> tasks on some of the VMs.
>>
>> Make sure that only VMs that exist in ACS database, also exist on KVM, and
>> virsh destroy all others (perhaps disable zone, destroy any running system
>> VMs and then make sure KVM vs DB vm list iz in correct sync.)
>>
>> Vlan 100 you were using for Public VLAN? Not sure how is your cloudbr1
>> (used for public traffic) configured and what vlans etc you are using now.
>> How did you change vlan 100?
>>
>> Andrija
>>
>>
>> On Wed, Aug 28, 2019, 19:32 Fariborz Navidan <md...@gmail.com>
>> wrote:
>>
>> > Any idea? I have also already managed devpts issue which causes issues
>> on
>> > CentOS 8. I have copied devpts line from /etc/fstab from another host.
>> Also
>> > SELinux is already disabled
>> >
>> > On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <mdvlinquest@gmail.com
>> >
>> > wrote:
>> >
>> > > This is agent log at debug log level:
>> > >
>> > > [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
>> > >
>> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
>> (Agent-Handler-2:null)
>> > > (logid:) Received response: Seq 1-1: { Ans: , MgmtId:
>> 157082893985413,
>> > > via: -1, Ver: v1, Flags: 100000,
>> > >
>> >
>> [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
>> (Agent-Handler-2:null)
>> > > (logid:) Startup task cancelled
>> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
>> (Agent-Handler-2:null)
>> > > (logid:) Proccess agent startup answer, agent id = 0
>> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
>> (Agent-Handler-2:null)
>> > > (logid:) Set agent id 0
>> > > 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent]
>> (Agent-Handler-2:null)
>> > > (logid:) Adding a watch list
>> > > 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent]
>> (Agent-Handler-2:null)
>> > > (logid:) Startup Response Received: agent id = 0
>> > > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (UgentTask-1:null) (logid:) Executing:
>> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
>> > > get_rule_logs_for_vms
>> > > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (UgentTask-1:null) (logid:) Executing while with timeout : 1800000
>> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100011,
>> > >
>> >
>> [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.CleanupNetworkRulesCmd
>> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
>> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
>> 1-4941856166108987393:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
>> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> > > 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
>> > >
>> > >
>> >
>> jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
>> > > 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to scan
>> 1
>> > > urls, producing 5 keys and 147 values
>> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
>> > > java.lang.Comparable -> java.lang.Enum
>> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
>> > > java.io.Serializable -> java.lang.Enum
>> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100111,
>> > >
>> >
>> [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.CheckNetworkCommand
>> > > 2019-08-28 18:59:16,327 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (UgentTask-1:null) (logid:) Execution is successful.
>> > > 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
>> > > (UgentTask-1:null) (logid:) Looking for libvirtd connection at:
>> > > qemu:///system
>> > > 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null)
>> > > (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver: v1,
>> > > Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,334 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing:
>> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
>> > > cleanup_rules
>> > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831)
>> matchPifFileInDirectory:
>> > > file name 'eth1.100'
>> > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing while with timeout :
>> > > 1800000
>> > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831)
>> matchPifFileInDirectory:
>> > > file name 'eth1'
>> > > 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
>> 1-4941856166108987394:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
>> > >
>> >
>> [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,396 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
>> > > 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent]
>> (Agent-Handler-4:null)
>> > > (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: ,
>> MgmtId:
>> > > 157082893985413, via: 1, Ver: v1, Flags: 10,
>> > >
>> [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
>> > > 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent]
>> (Agent-Handler-1:null)
>> > > (logid:) Received response: Seq 0-2: { Ans: , MgmtId:
>> 157082893985413,
>> > > via: 1, Ver: v1, Flags: 100010,
>> > >
>> >
>> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100011,
>> > >
>> >
>> [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.ModifyStoragePoolCommand
>> > > 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create
>> > storage
>> > > pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in
>> libvirt
>> > > 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
>> > > connection at: qemu:///system
>> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing defined
>> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
>> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch storage
>> > pool
>> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
>> > > 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
>> > > connection at: qemu:///system
>> > > 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed
>> pool
>> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
>> > 727711744
>> > > Available: 984168267776
>> > > 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
>> 1-4941856166108987395:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
>> > >
>> >
>> [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100111,
>> > >
>> >
>> [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
>> > > }
>> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100011,
>> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
>> > }
>> > > 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.ModifySshKeysCommand
>> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.SetHostParamsCommand
>> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Seq
>> 1-4941856166108987397:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
>> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100011,
>> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
>> > }
>> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.ModifySshKeysCommand
>> > > 2019-08-28 18:59:16,529 DEBUG
>> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
>> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600
>> > > /root/.ssh/id_rsa.cloud
>> > > 2019-08-28 18:59:16,529 DEBUG
>> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600
>> > > /root/.ssh/id_rsa.cloud
>> > > 2019-08-28 18:59:16,531 DEBUG
>> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
>> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with
>> > timeout
>> > > : 1800000
>> > > 2019-08-28 18:59:16,532 DEBUG
>> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with
>> > timeout
>> > > : 1800000
>> > > 2019-08-28 18:59:16,534 DEBUG
>> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
>> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is
>> successful.
>> > > 2019-08-28 18:59:16,534 DEBUG
>> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is
>> successful.
>> > > 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
>> 1-4941856166108987398:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
>> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> > > 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Seq
>> 1-4941856166108987396:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
>> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100111,
>> > > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}]
>> }
>> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.ReadyCommand
>> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent ready
>> > > command, agent id = 1
>> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
>> > > 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is
>> processed
>> > > for agent id = 1
>> > > 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
>> 1-4941856166108987399:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
>> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
>> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
>> > > 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
>> v1,
>> > > Flags: 100111,
>> > > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}]
>> }
>> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
>> > > com.cloud.agent.api.ReadyCommand
>> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent ready
>> > > command, agent id = 1
>> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
>> > > 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is
>> processed
>> > > for agent id = 1
>> > > 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
>> 1-4941856166108987400:
>> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
>> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
>> > > 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
>> > > com.cloud.agent.api.GetHostStatsCommand
>> > > 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
>> > > com.cloud.agent.api.GetStorageStatsCommand
>> > > 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch storage
>> > pool
>> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
>> > > 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
>> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd
>> > > connection at: qemu:///system
>> > > 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to
>> refresh
>> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
>> > > 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed
>> pool
>> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
>> > 727711744
>> > > Available: 984168267776
>> > > 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
>> > > (Thread-8:null) (logid:) Looking for libvirtd connection at:
>> > qemu:///system
>> > > 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor]
>> (Thread-8:null)
>> > > (logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
>> in
>> > > libvirt, continuing
>> > > 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor]
>> (Thread-8:null)
>> > > (logid:) Executing:
>> > >
>> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
>> > > 192.168.0.2 -p /home/primary1 -m
>> > /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
>> > > -h 192.168.0.2
>> > > 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor]
>> (Thread-8:null)
>> > > (logid:) Executing while with timeout : 60000
>> > > 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor]
>> (Thread-8:null)
>> > > (logid:) Execution is successful.
>> > > 2019-08-28 19:00:16,278 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (UgentTask-2:null) (logid:) Executing:
>> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
>> > > get_rule_logs_for_vms
>> > > 2019-08-28 19:00:16,281 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (UgentTask-2:null) (logid:) Executing while with timeout : 1800000
>> > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtComputingResource]
>> > > (UgentTask-2:null) (logid:) Execution is successful.
>> > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
>> > > (UgentTask-2:null) (logid:) Looking for libvirtd connection at:
>> > > qemu:///system
>> > > 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null)
>> > > (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver: v1,
>> > > Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
>> > > }
>> > > 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent]
>> (Agent-Handler-4:null)
>> > > (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
>> > > 157082893985413, via: 1, Ver: v1, Flags: 100010,
>> > >
>> >
>> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
>> > > }
>> > > 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
>> > > com.cloud.agent.api.GetHostStatsCommand
>> > > 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
>> > > (agentRequest-Handler-3:null) (logid:79632973) Processing command:
>> > > com.cloud.agent.api.GetStorageStatsCommand
>> > > 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch storage
>> > pool
>> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
>> > > 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
>> > > (agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd
>> > > connection at: qemu:///system
>> > > 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to
>> refresh
>> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
>> > > 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
>> > > (agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed
>> pool
>> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
>> > 727711744
>> > > Available: 984168267776
>> > > [root@fr-kvm1 primary1]#
>> > >
>> > > On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <
>> andrija.panic@gmail.com>
>> > > wrote:
>> > >
>> > >> I Don't see anything attached from agent.log? Make sure to put agent
>> in
>> > >> debug mode before posting logs (google for "cloudstack agent debug
>> > mode",
>> > >> there is a simple sed command, restart agent afterwards).
>> > >>
>> > >> Andrija
>> > >>
>> > >> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <md...@gmail.com>
>> > >> wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > Also this is agent log
>> > >> >
>> > >> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <
>> > mdvlinquest@gmail.com
>> > >> >
>> > >> > wrote:
>> > >> >
>> > >> > > Any idea?
>> > >> > >
>> > >> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
>> > >> mdvlinquest@gmail.com>
>> > >> > > wrote:
>> > >> > >
>> > >> > >> Thanks. I changed VLAN ID and now they are kept starting for
>> along
>> > >> time.
>> > >> > >> Latest logs:
>> > >> > >>
>> > >> > >>
>> > >> > >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been
>> > >> pending
>> > >> > for
>> > >> > >> 959 seconds
>> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been
>> > >> pending
>> > >> > for
>> > >> > >> 357 seconds
>> > >> > >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:34:14,852 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
>> > >> routers to
>> > >> > >> update status.
>> > >> > >> 2019-08-28 14:34:14,854 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
>> VPC
>> > >> > networks
>> > >> > >> to update Redundant State.
>> > >> > >> 2019-08-28 14:34:14,857 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
>> > >> networks
>> > >> > to
>> > >> > >> update RvR status.
>> > >> > >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598229
>> > >> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598229
>> > >> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
>> > Resetting
>> > >> > hosts
>> > >> > >> suitable for reconnect
>> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
>> > Completed
>> > >> > >> resetting hosts suitable for reconnect
>> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
>> > Acquiring
>> > >> > hosts
>> > >> > >> for clusters already owned by this management server
>> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
>> > Completed
>> > >> > >> acquiring hosts for clusters already owned by this management
>> > server
>> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
>> > Acquiring
>> > >> > hosts
>> > >> > >> for clusters not owned by any management server
>> > >> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
>> > Completed
>> > >> > >> acquiring hosts for clusters not owned by any management server
>> > >> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
>> > >> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
>> > >> > >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
>> > >> fr-kvm1.r9host.com)
>> > >> > >> 2019-08-28 14:34:24,974 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state
>> > report
>> > >> > from
>> > >> > >> ping process. host: 1
>> > >> > >> 2019-08-28 14:34:24,975 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Process VM state report.
>> > >> host:
>> > >> > 1,
>> > >> > >> number of records in report: 1
>> > >> > >> 2019-08-28 14:34:24,975 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host:
>> 1,
>> > vm
>> > >> id:
>> > >> > >> 103, power state: PowerOn
>> > >> > >> 2019-08-28 14:34:24,976 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
>> updated.
>> > >> > host:
>> > >> > >> 1, vm id: 103, power state: PowerOn
>> > >> > >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or
>> HA
>> > >> tasks
>> > >> > >> working on the VM. vm id: 103, postpone power-change report by
>> > >> resetting
>> > >> > >> power-change counters
>> > >> > >> 2019-08-28 14:34:24,980 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report.
>> > >> current
>> > >> > >> time: 1566995664980
>> > >> > >> 2019-08-28 14:34:24,980 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
>> host:
>> > >> 1, vm
>> > >> > >> id: 101, power state: PowerReportMissing, last state update:
>> > >> > 1566995544000
>> > >> > >> 2019-08-28 14:34:24,980 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since
>> > last
>> > >> > state
>> > >> > >> update(120980ms) has passed graceful period
>> > >> > >> 2019-08-28 14:34:24,982 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
>> updated.
>> > >> > host:
>> > >> > >> 1, vm id: 101, power state: PowerReportMissing
>> > >> > >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or
>> HA
>> > >> tasks
>> > >> > >> working on the VM. vm id: 101, postpone power-change report by
>> > >> resetting
>> > >> > >> power-change counters
>> > >> > >> 2019-08-28 14:34:24,984 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
>> host:
>> > >> 1, vm
>> > >> > >> id: 102, power state: PowerReportMissing, last state update:
>> > >> > 1566995544000
>> > >> > >> 2019-08-28 14:34:24,984 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since
>> > last
>> > >> > state
>> > >> > >> update(120980ms) has passed graceful period
>> > >> > >> 2019-08-28 14:34:24,986 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
>> updated.
>> > >> > host:
>> > >> > >> 1, vm id: 102, power state: PowerReportMissing
>> > >> > >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or
>> HA
>> > >> tasks
>> > >> > >> working on the VM. vm id: 102, postpone power-change report by
>> > >> resetting
>> > >> > >> power-change counters
>> > >> > >> 2019-08-28 14:34:24,987 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-15:null) (logid:) Done with process of VM
>> > state
>> > >> > >> report. host: 1
>> > >> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598231
>> > >> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598231
>> > >> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
>> > >> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
>> > >> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling
>> > Monitor
>> > >> is
>> > >> > >> running...
>> > >> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
>> > >> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
>> > >> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
>> > >> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
>> > >> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
>> > >> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
>> Api
>> > >> > >> Discovery plugin was unable to find an api by that name or
>> process
>> > >> any
>> > >> > apis
>> > >> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
>> > >> > >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
>> > >> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
>> > >> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07)
>> HostStatsCollector
>> > >> is
>> > >> > >> running...
>> > >> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
>> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
>> > >> > 1-5778962746846479041:
>> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> > >> fr-kvm1.r9host.com
>> > >> > ),
>> > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
>> StorageCollector
>> > is
>> > >> > >> running...
>> > >> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no
>> > >> secondary
>> > >> > >> storage VM for secondary storage host Secondary1
>> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
>> > >> > getCommandHostDelegation:
>> > >> > >> class com.cloud.agent.api.GetStorageStatsCommand
>> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
>> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are
>> returning
>> > the
>> > >> > >> default host to execute commands because the command is not of
>> Copy
>> > >> > type.
>> > >> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
>> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
>> > >> > 1-5778962746846479042:
>> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> > >> fr-kvm1.r9host.com
>> > >> > ),
>> > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> > >> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
>> > >> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
>> > >> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
>> > >> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
>> > >> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
>> > >> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
>> Api
>> > >> > >> Discovery plugin was unable to find an api by that name or
>> process
>> > >> any
>> > >> > apis
>> > >> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
>> > >> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598240
>> > >> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598240
>> > >> > >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:34:44,852 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
>> > >> routers to
>> > >> > >> update status.
>> > >> > >> 2019-08-28 14:34:44,855 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
>> VPC
>> > >> > networks
>> > >> > >> to update Redundant State.
>> > >> > >> 2019-08-28 14:34:44,857 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
>> > >> networks
>> > >> > to
>> > >> > >> update RvR status.
>> > >> > >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
>> > >> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:48,215 WARN
>> > [c.c.a.d.ParamGenericValidationWorker]
>> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
>> > >> Received
>> > >> > >> unknown parameters for command listSystemVms. Unknown
>> parameters :
>> > >> > listall
>> > >> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
>> > >> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
>> > >> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> > >> > >>>Searching
>> > >> > >> for hosts>>>
>> > >> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> > >> > >> >>>Generating Response>>>
>> > >> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
>> > >> > >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
>> > >> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
>> > >> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
>> > >> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
>> > >> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598245
>> > >> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598245
>> > >> > >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
>> > >> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
>> > >> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
>> > >> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
>> > >> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
>> > >> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
>> > >> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
>> > >> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
>> Api
>> > >> > >> Discovery plugin was unable to find an api by that name or
>> process
>> > >> any
>> > >> > apis
>> > >> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
>> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been
>> > >> pending
>> > >> > for
>> > >> > >> 1019 seconds
>> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been
>> > >> pending
>> > >> > for
>> > >> > >> 417 seconds
>> > >> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
>> > >> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
>> > >> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
>> > >> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started
>> > >> resource
>> > >> > >> counters recalculation periodic task.
>> > >> > >> 2019-08-28 14:35:14,848 DEBUG
>> > [c.c.n.ExternalDeviceUsageManagerImpl]
>> > >> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32)
>> External
>> > >> > devices
>> > >> > >> are not used. Skipping external device usage collection
>> > >> > >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
>> > >> > scheduler.poll is
>> > >> > >> being called at 2019-08-28 12:35:14 GMT
>> > >> > >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:35:14,857 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
>> > >> routers to
>> > >> > >> update status.
>> > >> > >> 2019-08-28 14:35:14,857 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > (RouterMonitor-1:ctx-7cb475c8)
>> > >> > >> (logid:cc3e421d) Found 0 running routers.
>> > >> > >> 2019-08-28 14:35:14,858 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
>> VPC
>> > >> > networks
>> > >> > >> to update Redundant State.
>> > >> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0
>> snapshots to
>> > >> be
>> > >> > >> executed at 2019-08-28 12:35:14 GMT
>> > >> > >> 2019-08-28 14:35:14,858 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
>> > >> networks
>> > >> > to
>> > >> > >> update RvR status.
>> > >> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
>> > >> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
>> > >> > NetworkGarbageCollector
>> > >> > >> uses '600' seconds for GC interval.
>> > >> > >> 2019-08-28 14:35:14,925 INFO
>> [c.c.h.HighAvailabilityManagerImpl]
>> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage
>> > server
>> > >> > >> 2019-08-28 14:35:14,928 DEBUG
>> [c.c.h.HighAvailabilityManagerImpl]
>> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true,
>> > >> > heartbeat:
>> > >> > >> Wed Aug 28 14:35:12 CEST 2019
>> > >> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
>> > >> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
>> > >> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
>> > >> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
>> > >> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
>> > >> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
>> Api
>> > >> > >> Discovery plugin was unable to find an api by that name or
>> process
>> > >> any
>> > >> > apis
>> > >> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
>> > >> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598254
>> > >> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listInfrastructure&response=json&_=1566995598254
>> > >> > >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>> > >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
>> > >> Manager
>> > >> > is
>> > >> > >> running and getting the updates from LB providers and updating
>> > >> service
>> > >> > >> status
>> > >> > >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>> > >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
>> > >> Manager
>> > >> > is
>> > >> > >> running and getting the updates from LB providers and updating
>> > >> service
>> > >> > >> status
>> > >> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Ping from 1(
>> > >> fr-kvm1.r9host.com)
>> > >> > >> 2019-08-28 14:35:25,005 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Process host VM state
>> report
>> > >> from
>> > >> > >> ping process. host: 1
>> > >> > >> 2019-08-28 14:35:25,006 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Process VM state report.
>> > >> host: 1,
>> > >> > >> number of records in report: 1
>> > >> > >> 2019-08-28 14:35:25,006 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) VM state report. host:
>> 1, vm
>> > >> id:
>> > >> > >> 103, power state: PowerOn
>> > >> > >> 2019-08-28 14:35:25,007 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) VM state report is
>> updated.
>> > >> > host: 1,
>> > >> > >> vm id: 103, power state: PowerOn
>> > >> > >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) There is pending job or
>> HA
>> > >> tasks
>> > >> > >> working on the VM. vm id: 103, postpone power-change report by
>> > >> resetting
>> > >> > >> power-change counters
>> > >> > >> 2019-08-28 14:35:25,011 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Run missing VM report.
>> > current
>> > >> > time:
>> > >> > >> 1566995725011
>> > >> > >> 2019-08-28 14:35:25,012 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM.
>> host:
>> > 1,
>> > >> vm
>> > >> > id:
>> > >> > >> 101, power state: PowerReportMissing, last state update:
>> > >> 1566995664000
>> > >> > >> 2019-08-28 14:35:25,012 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since
>> last
>> > >> > state
>> > >> > >> update(61011ms) has not passed graceful period yet
>> > >> > >> 2019-08-28 14:35:25,012 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM.
>> host:
>> > 1,
>> > >> vm
>> > >> > id:
>> > >> > >> 102, power state: PowerReportMissing, last state update:
>> > >> 1566995664000
>> > >> > >> 2019-08-28 14:35:25,012 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since
>> last
>> > >> > state
>> > >> > >> update(61011ms) has not passed graceful period yet
>> > >> > >> 2019-08-28 14:35:25,012 DEBUG
>> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> > >> (AgentManager-Handler-1:null) (logid:) Done with process of VM
>> > state
>> > >> > >> report. host: 1
>> > >> > >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
>> > >> > >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:26,390 WARN
>> > [c.c.a.d.ParamGenericValidationWorker]
>> > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
>> > >> Received
>> > >> > >> unknown parameters for command listSystemVms. Unknown
>> parameters :
>> > >> > listall
>> > >> > >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
>> > >> > >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listHosts&response=json&details=min&_=1566995598256
>> > >> > >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> > >> > >>>Searching
>> > >> > >> for hosts>>>
>> > >> > >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> > >> > >> >>>Generating Response>>>
>> > >> > >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >> command=listHosts&response=json&details=min&_=1566995598256
>> > >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
>> > >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq
>> > >> > 1-5778962746846479043:
>> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> > >> fr-kvm1.r9host.com
>> > >> > ),
>> > >> > >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
>> > >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from
>> > >> executing
>> > >> > >> class com.cloud.agent.api.GetVolumeStatsCommand:
>> > >> > >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling
>> > Monitor
>> > >> is
>> > >> > >> running...
>> > >> > >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
>> > >> > >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
>> > >> > >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
>> > >> > >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
>> > >> > >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325)
>> HostStatsCollector
>> > >> is
>> > >> > >> running...
>> > >> > >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
>> > >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq
>> > >> > 1-5778962746846479044:
>> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> > >> fr-kvm1.r9host.com
>> > >> > ),
>> > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> > >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
>> StorageCollector
>> > is
>> > >> > >> running...
>> > >> > >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
>> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no
>> > >> secondary
>> > >> > >> storage VM for secondary storage host Secondary1
>> > >> > >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
>> > >> > getCommandHostDelegation:
>> > >> > >> class com.cloud.agent.api.GetStorageStatsCommand
>> > >> > >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
>> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are
>> returning
>> > the
>> > >> > >> default host to execute commands because the command is not of
>> Copy
>> > >> > type.
>> > >> > >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
>> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq
>> > >> > 1-5778962746846479045:
>> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> > >> fr-kvm1.r9host.com
>> > >> > ),
>> > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> > >> > >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
>> > >> > >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
>> > >> > >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
>> > >> > >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
>> > >> > >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
>> > >> > >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
>> > >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
>> > CIDRs
>> > >> > from
>> > >> > >> which account
>> 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >> > allowed
>> > >> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> > >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
>> > >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
>> > >> ===END===
>> > >> > >> 137.74.35.102 -- GET
>> > >> > >>
>> > >> >
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
>> > >> > >> 2019-08-28 14:35:44,845 DEBUG
>> [c.c.d.DeploymentPlanningManagerImpl]
>> > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
>> > >> Checking
>> > >> > if
>> > >> > >> any host reservation can be released ...
>> > >> > >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin
>> > cleanup
>> > >> > >> expired async-jobs
>> > >> > >> 2019-08-28 14:35:44,852 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
>> > >> routers to
>> > >> > >> update status.
>> > >> > >> 2019-08-28 14:35:44,854 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
>> VPC
>> > >> > networks
>> > >> > >> to update Redundant State.
>> > >> > >> 2019-08-28 14:35:44,856 DEBUG
>> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
>> > >> networks
>> > >> > to
>> > >> > >> update RvR status.
>> > >> > >> 2019-08-28 14:35:44,856 DEBUG
>> [c.c.d.DeploymentPlanningManagerImpl]
>> > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
>> > Cannot
>> > >> > >> release reservation, Found 3 VMs Running on host 1
>> > >> > >> 2019-08-28 14:35:44,856 DEBUG
>> [c.c.d.DeploymentPlanningManagerImpl]
>> > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
>> Done
>> > >> > running
>> > >> > >> HostReservationReleaseChecker ...
>> > >> > >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End
>> cleanup
>> > >> > expired
>> > >> > >> async-jobs
>> > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity
>> > >> Checker
>> > >> > >> ...
>> > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating
>> > system
>> > >> > >> capacity
>> > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing
>> cpu/ram
>> > >> > capacity
>> > >> > >> update
>> > >> > >
>>
>
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
It is CentOS Linux release 7.6.1810 (Core) Agent is misleading. virsh says
no domain is in KVM.
[root@fr-kvm1 ~]# virsh list --all
Id Name State
----------------------------------------------------
[root@fr-kvm1 ~]#
On Wed, Aug 28, 2019 at 10:46 PM Andrija Panic <an...@gmail.com>
wrote:
> That is brave :) I mean running unsupported CentOS8 with ACS. Seriously, if
> not mistaken, we plan to support it in 4.14 only, and this might be causing
> issues as well.
>
> Your KVM host reports its already running 3 VMs, and that there are pending
> tasks on some of the VMs.
>
> Make sure that only VMs that exist in ACS database, also exist on KVM, and
> virsh destroy all others (perhaps disable zone, destroy any running system
> VMs and then make sure KVM vs DB vm list iz in correct sync.)
>
> Vlan 100 you were using for Public VLAN? Not sure how is your cloudbr1
> (used for public traffic) configured and what vlans etc you are using now.
> How did you change vlan 100?
>
> Andrija
>
>
> On Wed, Aug 28, 2019, 19:32 Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Any idea? I have also already managed devpts issue which causes issues on
> > CentOS 8. I have copied devpts line from /etc/fstab from another host.
> Also
> > SELinux is already disabled
> >
> > On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> > > This is agent log at debug log level:
> > >
> > > [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
> > >
> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
> (Agent-Handler-2:null)
> > > (logid:) Received response: Seq 1-1: { Ans: , MgmtId: 157082893985413,
> > > via: -1, Ver: v1, Flags: 100000,
> > >
> >
> [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent]
> (Agent-Handler-2:null)
> > > (logid:) Startup task cancelled
> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
> (Agent-Handler-2:null)
> > > (logid:) Proccess agent startup answer, agent id = 0
> > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent]
> (Agent-Handler-2:null)
> > > (logid:) Set agent id 0
> > > 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent]
> (Agent-Handler-2:null)
> > > (logid:) Adding a watch list
> > > 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent]
> (Agent-Handler-2:null)
> > > (logid:) Startup Response Received: agent id = 0
> > > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (UgentTask-1:null) (logid:) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > get_rule_logs_for_vms
> > > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (UgentTask-1:null) (logid:) Executing while with timeout : 1800000
> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100011,
> > >
> >
> [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.CleanupNetworkRulesCmd
> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
> > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
> 1-4941856166108987393:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > > 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
> > > (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
> > >
> > >
> >
> jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
> > > 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
> > > (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to scan 1
> > > urls, producing 5 keys and 147 values
> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> > > java.lang.Comparable -> java.lang.Enum
> > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> > > java.io.Serializable -> java.lang.Enum
> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100111,
> > >
> >
> [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.CheckNetworkCommand
> > > 2019-08-28 18:59:16,327 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (UgentTask-1:null) (logid:) Execution is successful.
> > > 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
> > > (UgentTask-1:null) (logid:) Looking for libvirtd connection at:
> > > qemu:///system
> > > 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null)
> > > (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver: v1,
> > > Flags: 11,
> > >
> >
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,334 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > cleanup_rules
> > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
> > > file name 'eth1.100'
> > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (Agent-Handler-4:null) (logid:1f7ef831) Executing while with timeout :
> > > 1800000
> > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
> > > file name 'eth1'
> > > 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
> 1-4941856166108987394:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > >
> >
> [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,396 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
> > > 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent]
> (Agent-Handler-4:null)
> > > (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: ,
> MgmtId:
> > > 157082893985413, via: 1, Ver: v1, Flags: 10,
> > > [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}]
> }
> > > 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent]
> (Agent-Handler-1:null)
> > > (logid:) Received response: Seq 0-2: { Ans: , MgmtId: 157082893985413,
> > > via: 1, Ver: v1, Flags: 100010,
> > >
> >
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100011,
> > >
> >
> [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.ModifyStoragePoolCommand
> > > 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create
> > storage
> > > pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in
> libvirt
> > > 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> > > connection at: qemu:///system
> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing defined
> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
> > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch storage
> > pool
> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > > 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> > > connection at: qemu:///system
> > > 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed
> pool
> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> > 727711744
> > > Available: 984168267776
> > > 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
> 1-4941856166108987395:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > >
> >
> [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100111,
> > >
> >
> [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
> > > }
> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100011,
> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> > }
> > > 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.ModifySshKeysCommand
> > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.SetHostParamsCommand
> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-1:null) (logid:1f7ef831) Seq
> 1-4941856166108987397:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100011,
> [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> > }
> > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.ModifySshKeysCommand
> > > 2019-08-28 18:59:16,529 DEBUG
> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600
> > > /root/.ssh/id_rsa.cloud
> > > 2019-08-28 18:59:16,529 DEBUG
> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600
> > > /root/.ssh/id_rsa.cloud
> > > 2019-08-28 18:59:16,531 DEBUG
> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with
> > timeout
> > > : 1800000
> > > 2019-08-28 18:59:16,532 DEBUG
> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with
> > timeout
> > > : 1800000
> > > 2019-08-28 18:59:16,534 DEBUG
> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is successful.
> > > 2019-08-28 18:59:16,534 DEBUG
> > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is successful.
> > > 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq
> 1-4941856166108987398:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > > 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-5:null) (logid:1f7ef831) Seq
> 1-4941856166108987396:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100111,
> > > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
> > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.ReadyCommand
> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent ready
> > > command, agent id = 1
> > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
> > > 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is
> processed
> > > for agent id = 1
> > > 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq
> 1-4941856166108987399:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> > > 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1, Ver:
> v1,
> > > Flags: 100111,
> > > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
> > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> > > com.cloud.agent.api.ReadyCommand
> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent ready
> > > command, agent id = 1
> > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
> > > 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is
> processed
> > > for agent id = 1
> > > 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq
> 1-4941856166108987400:
> > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> > > 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
> > > com.cloud.agent.api.GetHostStatsCommand
> > > 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
> > > com.cloud.agent.api.GetStorageStatsCommand
> > > 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch storage
> > pool
> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > > 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd
> > > connection at: qemu:///system
> > > 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to
> refresh
> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> > > 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed
> pool
> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> > 727711744
> > > Available: 984168267776
> > > 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
> > > (Thread-8:null) (logid:) Looking for libvirtd connection at:
> > qemu:///system
> > > 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor]
> (Thread-8:null)
> > > (logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 in
> > > libvirt, continuing
> > > 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor]
> (Thread-8:null)
> > > (logid:) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> -i
> > > 192.168.0.2 -p /home/primary1 -m
> > /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
> > > -h 192.168.0.2
> > > 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor]
> (Thread-8:null)
> > > (logid:) Executing while with timeout : 60000
> > > 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor]
> (Thread-8:null)
> > > (logid:) Execution is successful.
> > > 2019-08-28 19:00:16,278 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (UgentTask-2:null) (logid:) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > get_rule_logs_for_vms
> > > 2019-08-28 19:00:16,281 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (UgentTask-2:null) (logid:) Executing while with timeout : 1800000
> > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (UgentTask-2:null) (logid:) Execution is successful.
> > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
> > > (UgentTask-2:null) (logid:) Looking for libvirtd connection at:
> > > qemu:///system
> > > 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null)
> > > (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver: v1,
> > > Flags: 11,
> > >
> >
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
> > > }
> > > 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent]
> (Agent-Handler-4:null)
> > > (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
> > > 157082893985413, via: 1, Ver: v1, Flags: 100010,
> > >
> >
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
> > > }
> > > 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
> > > com.cloud.agent.api.GetHostStatsCommand
> > > 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-3:null) (logid:79632973) Processing command:
> > > com.cloud.agent.api.GetStorageStatsCommand
> > > 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch storage
> > pool
> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > > 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
> > > (agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd
> > > connection at: qemu:///system
> > > 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to
> refresh
> > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> > > 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > > (agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed
> pool
> > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> > 727711744
> > > Available: 984168267776
> > > [root@fr-kvm1 primary1]#
> > >
> > > On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <andrija.panic@gmail.com
> >
> > > wrote:
> > >
> > >> I Don't see anything attached from agent.log? Make sure to put agent
> in
> > >> debug mode before posting logs (google for "cloudstack agent debug
> > mode",
> > >> there is a simple sed command, restart agent afterwards).
> > >>
> > >> Andrija
> > >>
> > >> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <md...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > Also this is agent log
> > >> >
> > >> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <
> > mdvlinquest@gmail.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > Any idea?
> > >> > >
> > >> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
> > >> mdvlinquest@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > >> Thanks. I changed VLAN ID and now they are kept starting for
> along
> > >> time.
> > >> > >> Latest logs:
> > >> > >>
> > >> > >>
> > >> > >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been
> > >> pending
> > >> > for
> > >> > >> 959 seconds
> > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been
> > >> pending
> > >> > for
> > >> > >> 357 seconds
> > >> > >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:34:14,852 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> > >> routers to
> > >> > >> update status.
> > >> > >> 2019-08-28 14:34:14,854 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC
> > >> > networks
> > >> > >> to update Redundant State.
> > >> > >> 2019-08-28 14:34:14,857 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> > >> networks
> > >> > to
> > >> > >> update RvR status.
> > >> > >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598229
> > >> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598229
> > >> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > Resetting
> > >> > hosts
> > >> > >> suitable for reconnect
> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > Completed
> > >> > >> resetting hosts suitable for reconnect
> > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > Acquiring
> > >> > hosts
> > >> > >> for clusters already owned by this management server
> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > Completed
> > >> > >> acquiring hosts for clusters already owned by this management
> > server
> > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > Acquiring
> > >> > hosts
> > >> > >> for clusters not owned by any management server
> > >> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
> > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> > Completed
> > >> > >> acquiring hosts for clusters not owned by any management server
> > >> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> > >> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> > >> > >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
> > >> fr-kvm1.r9host.com)
> > >> > >> 2019-08-28 14:34:24,974 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state
> > report
> > >> > from
> > >> > >> ping process. host: 1
> > >> > >> 2019-08-28 14:34:24,975 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Process VM state report.
> > >> host:
> > >> > 1,
> > >> > >> number of records in report: 1
> > >> > >> 2019-08-28 14:34:24,975 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1,
> > vm
> > >> id:
> > >> > >> 103, power state: PowerOn
> > >> > >> 2019-08-28 14:34:24,976 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> updated.
> > >> > host:
> > >> > >> 1, vm id: 103, power state: PowerOn
> > >> > >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or
> HA
> > >> tasks
> > >> > >> working on the VM. vm id: 103, postpone power-change report by
> > >> resetting
> > >> > >> power-change counters
> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report.
> > >> current
> > >> > >> time: 1566995664980
> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
> host:
> > >> 1, vm
> > >> > >> id: 101, power state: PowerReportMissing, last state update:
> > >> > 1566995544000
> > >> > >> 2019-08-28 14:34:24,980 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since
> > last
> > >> > state
> > >> > >> update(120980ms) has passed graceful period
> > >> > >> 2019-08-28 14:34:24,982 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> updated.
> > >> > host:
> > >> > >> 1, vm id: 101, power state: PowerReportMissing
> > >> > >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or
> HA
> > >> tasks
> > >> > >> working on the VM. vm id: 101, postpone power-change report by
> > >> resetting
> > >> > >> power-change counters
> > >> > >> 2019-08-28 14:34:24,984 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM.
> host:
> > >> 1, vm
> > >> > >> id: 102, power state: PowerReportMissing, last state update:
> > >> > 1566995544000
> > >> > >> 2019-08-28 14:34:24,984 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since
> > last
> > >> > state
> > >> > >> update(120980ms) has passed graceful period
> > >> > >> 2019-08-28 14:34:24,986 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is
> updated.
> > >> > host:
> > >> > >> 1, vm id: 102, power state: PowerReportMissing
> > >> > >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or
> HA
> > >> tasks
> > >> > >> working on the VM. vm id: 102, postpone power-change report by
> > >> resetting
> > >> > >> power-change counters
> > >> > >> 2019-08-28 14:34:24,987 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-15:null) (logid:) Done with process of VM
> > state
> > >> > >> report. host: 1
> > >> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598231
> > >> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598231
> > >> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> > >> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> > >> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling
> > Monitor
> > >> is
> > >> > >> running...
> > >> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> > >> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> > >> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> > >> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> > >> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> > >> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> Api
> > >> > >> Discovery plugin was unable to find an api by that name or
> process
> > >> any
> > >> > apis
> > >> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> > >> > >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> > >> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> > >> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07)
> HostStatsCollector
> > >> is
> > >> > >> running...
> > >> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
> > >> > 1-5778962746846479041:
> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> fr-kvm1.r9host.com
> > >> > ),
> > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector
> > is
> > >> > >> running...
> > >> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no
> > >> secondary
> > >> > >> storage VM for secondary storage host Secondary1
> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> > >> > getCommandHostDelegation:
> > >> > >> class com.cloud.agent.api.GetStorageStatsCommand
> > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning
> > the
> > >> > >> default host to execute commands because the command is not of
> Copy
> > >> > type.
> > >> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
> > >> > 1-5778962746846479042:
> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> fr-kvm1.r9host.com
> > >> > ),
> > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> > >> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> > >> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> > >> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> > >> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> > >> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> Api
> > >> > >> Discovery plugin was unable to find an api by that name or
> process
> > >> any
> > >> > apis
> > >> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> > >> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598240
> > >> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598240
> > >> > >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:34:44,852 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> > >> routers to
> > >> > >> update status.
> > >> > >> 2019-08-28 14:34:44,855 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC
> > >> > networks
> > >> > >> to update Redundant State.
> > >> > >> 2019-08-28 14:34:44,857 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> > >> networks
> > >> > to
> > >> > >> update RvR status.
> > >> > >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> > >> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:48,215 WARN
> > [c.c.a.d.ParamGenericValidationWorker]
> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> > >> Received
> > >> > >> unknown parameters for command listSystemVms. Unknown parameters
> :
> > >> > listall
> > >> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> > >> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
> > >> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> > >> > >>>Searching
> > >> > >> for hosts>>>
> > >> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> > >> > >> >>>Generating Response>>>
> > >> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listHosts&response=json&details=min&_=1566995598242
> > >> > >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> > >> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> > >> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> > >> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> > >> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598245
> > >> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598245
> > >> > >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> > >> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> > >> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> > >> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> > >> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> > >> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> > >> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> > >> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> Api
> > >> > >> Discovery plugin was unable to find an api by that name or
> process
> > >> any
> > >> > apis
> > >> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been
> > >> pending
> > >> > for
> > >> > >> 1019 seconds
> > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been
> > >> pending
> > >> > for
> > >> > >> 417 seconds
> > >> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> > >> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> > >> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
> > >> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started
> > >> resource
> > >> > >> counters recalculation periodic task.
> > >> > >> 2019-08-28 14:35:14,848 DEBUG
> > [c.c.n.ExternalDeviceUsageManagerImpl]
> > >> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External
> > >> > devices
> > >> > >> are not used. Skipping external device usage collection
> > >> > >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
> > >> > scheduler.poll is
> > >> > >> being called at 2019-08-28 12:35:14 GMT
> > >> > >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:35:14,857 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> > >> routers to
> > >> > >> update status.
> > >> > >> 2019-08-28 14:35:14,857 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > (RouterMonitor-1:ctx-7cb475c8)
> > >> > >> (logid:cc3e421d) Found 0 running routers.
> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC
> > >> > networks
> > >> > >> to update Redundant State.
> > >> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots
> to
> > >> be
> > >> > >> executed at 2019-08-28 12:35:14 GMT
> > >> > >> 2019-08-28 14:35:14,858 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> > >> networks
> > >> > to
> > >> > >> update RvR status.
> > >> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
> > >> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
> > >> > NetworkGarbageCollector
> > >> > >> uses '600' seconds for GC interval.
> > >> > >> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage
> > server
> > >> > >> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
> > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true,
> > >> > heartbeat:
> > >> > >> Wed Aug 28 14:35:12 CEST 2019
> > >> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> > >> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> > >> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> > >> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> > >> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> > >> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> Api
> > >> > >> Discovery plugin was unable to find an api by that name or
> process
> > >> any
> > >> > apis
> > >> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> > >> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598254
> > >> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listInfrastructure&response=json&_=1566995598254
> > >> > >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> > >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
> > >> Manager
> > >> > is
> > >> > >> running and getting the updates from LB providers and updating
> > >> service
> > >> > >> status
> > >> > >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> > >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
> > >> Manager
> > >> > is
> > >> > >> running and getting the updates from LB providers and updating
> > >> service
> > >> > >> status
> > >> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Ping from 1(
> > >> fr-kvm1.r9host.com)
> > >> > >> 2019-08-28 14:35:25,005 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Process host VM state
> report
> > >> from
> > >> > >> ping process. host: 1
> > >> > >> 2019-08-28 14:35:25,006 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Process VM state report.
> > >> host: 1,
> > >> > >> number of records in report: 1
> > >> > >> 2019-08-28 14:35:25,006 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1,
> vm
> > >> id:
> > >> > >> 103, power state: PowerOn
> > >> > >> 2019-08-28 14:35:25,007 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) VM state report is
> updated.
> > >> > host: 1,
> > >> > >> vm id: 103, power state: PowerOn
> > >> > >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) There is pending job or HA
> > >> tasks
> > >> > >> working on the VM. vm id: 103, postpone power-change report by
> > >> resetting
> > >> > >> power-change counters
> > >> > >> 2019-08-28 14:35:25,011 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Run missing VM report.
> > current
> > >> > time:
> > >> > >> 1566995725011
> > >> > >> 2019-08-28 14:35:25,012 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host:
> > 1,
> > >> vm
> > >> > id:
> > >> > >> 101, power state: PowerReportMissing, last state update:
> > >> 1566995664000
> > >> > >> 2019-08-28 14:35:25,012 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since
> last
> > >> > state
> > >> > >> update(61011ms) has not passed graceful period yet
> > >> > >> 2019-08-28 14:35:25,012 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host:
> > 1,
> > >> vm
> > >> > id:
> > >> > >> 102, power state: PowerReportMissing, last state update:
> > >> 1566995664000
> > >> > >> 2019-08-28 14:35:25,012 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since
> last
> > >> > state
> > >> > >> update(61011ms) has not passed graceful period yet
> > >> > >> 2019-08-28 14:35:25,012 DEBUG
> > >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> > >> (AgentManager-Handler-1:null) (logid:) Done with process of VM
> > state
> > >> > >> report. host: 1
> > >> > >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> > >> > >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:26,390 WARN
> > [c.c.a.d.ParamGenericValidationWorker]
> > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> > >> Received
> > >> > >> unknown parameters for command listSystemVms. Unknown parameters
> :
> > >> > listall
> > >> > >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> > >> > >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listHosts&response=json&details=min&_=1566995598256
> > >> > >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> > >> > >>>Searching
> > >> > >> for hosts>>>
> > >> > >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> > >> > >> >>>Generating Response>>>
> > >> > >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >> command=listHosts&response=json&details=min&_=1566995598256
> > >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
> > >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq
> > >> > 1-5778962746846479043:
> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> fr-kvm1.r9host.com
> > >> > ),
> > >> > >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
> > >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from
> > >> executing
> > >> > >> class com.cloud.agent.api.GetVolumeStatsCommand:
> > >> > >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling
> > Monitor
> > >> is
> > >> > >> running...
> > >> > >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> > >> > >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> > >> > >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> > >> > >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> > >> > >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325)
> HostStatsCollector
> > >> is
> > >> > >> running...
> > >> > >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
> > >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq
> > >> > 1-5778962746846479044:
> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> fr-kvm1.r9host.com
> > >> > ),
> > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> > >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector
> > is
> > >> > >> running...
> > >> > >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no
> > >> secondary
> > >> > >> storage VM for secondary storage host Secondary1
> > >> > >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
> > >> > getCommandHostDelegation:
> > >> > >> class com.cloud.agent.api.GetStorageStatsCommand
> > >> > >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning
> > the
> > >> > >> default host to execute commands because the command is not of
> Copy
> > >> > type.
> > >> > >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
> > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq
> > >> > 1-5778962746846479045:
> > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> > >> fr-kvm1.r9host.com
> > >> > ),
> > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >> > >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> > >> > >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> > >> > >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> > >> > >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> > >> > >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> > >> > >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
> > >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
> > CIDRs
> > >> > from
> > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]'
> is
> > >> > allowed
> > >> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> > >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
> > >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
> > >> ===END===
> > >> > >> 137.74.35.102 -- GET
> > >> > >>
> > >> >
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> > >> > >> 2019-08-28 14:35:44,845 DEBUG
> [c.c.d.DeploymentPlanningManagerImpl]
> > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
> > >> Checking
> > >> > if
> > >> > >> any host reservation can be released ...
> > >> > >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin
> > cleanup
> > >> > >> expired async-jobs
> > >> > >> 2019-08-28 14:35:44,852 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
> > >> routers to
> > >> > >> update status.
> > >> > >> 2019-08-28 14:35:44,854 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC
> > >> > networks
> > >> > >> to update Redundant State.
> > >> > >> 2019-08-28 14:35:44,856 DEBUG
> > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
> > >> networks
> > >> > to
> > >> > >> update RvR status.
> > >> > >> 2019-08-28 14:35:44,856 DEBUG
> [c.c.d.DeploymentPlanningManagerImpl]
> > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
> > Cannot
> > >> > >> release reservation, Found 3 VMs Running on host 1
> > >> > >> 2019-08-28 14:35:44,856 DEBUG
> [c.c.d.DeploymentPlanningManagerImpl]
> > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
> Done
> > >> > running
> > >> > >> HostReservationReleaseChecker ...
> > >> > >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End
> cleanup
> > >> > expired
> > >> > >> async-jobs
> > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity
> > >> Checker
> > >> > >> ...
> > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating
> > system
> > >> > >> capacity
> > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram
> > >> > capacity
> > >> > >> update
> > >> > >
>
Re: Unable to start System VMs
Posted by Andrija Panic <an...@gmail.com>.
That is brave :) I mean running unsupported CentOS8 with ACS. Seriously, if
not mistaken, we plan to support it in 4.14 only, and this might be causing
issues as well.
Your KVM host reports its already running 3 VMs, and that there are pending
tasks on some of the VMs.
Make sure that only VMs that exist in ACS database, also exist on KVM, and
virsh destroy all others (perhaps disable zone, destroy any running system
VMs and then make sure KVM vs DB vm list iz in correct sync.)
Vlan 100 you were using for Public VLAN? Not sure how is your cloudbr1
(used for public traffic) configured and what vlans etc you are using now.
How did you change vlan 100?
Andrija
On Wed, Aug 28, 2019, 19:32 Fariborz Navidan <md...@gmail.com> wrote:
> Any idea? I have also already managed devpts issue which causes issues on
> CentOS 8. I have copied devpts line from /etc/fstab from another host. Also
> SELinux is already disabled
>
> On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > This is agent log at debug log level:
> >
> > [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
> >
> > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
> > (logid:) Received response: Seq 1-1: { Ans: , MgmtId: 157082893985413,
> > via: -1, Ver: v1, Flags: 100000,
> >
> [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
> > }
> > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
> > (logid:) Startup task cancelled
> > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> > (logid:) Proccess agent startup answer, agent id = 0
> > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> > (logid:) Set agent id 0
> > 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
> > (logid:) Adding a watch list
> > 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> > (logid:) Startup Response Received: agent id = 0
> > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
> > (UgentTask-1:null) (logid:) Executing:
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > get_rule_logs_for_vms
> > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
> > (UgentTask-1:null) (logid:) Executing while with timeout : 1800000
> > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100011,
> >
> [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
> > }
> > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.CleanupNetworkRulesCmd
> > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
> > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq 1-4941856166108987393:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
> > (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
> >
> >
> jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
> > 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
> > (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to scan 1
> > urls, producing 5 keys and 147 values
> > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> > java.lang.Comparable -> java.lang.Enum
> > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> > java.io.Serializable -> java.lang.Enum
> > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100111,
> >
> [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
> > }
> > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.CheckNetworkCommand
> > 2019-08-28 18:59:16,327 DEBUG [kvm.resource.LibvirtComputingResource]
> > (UgentTask-1:null) (logid:) Execution is successful.
> > 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
> > (UgentTask-1:null) (logid:) Looking for libvirtd connection at:
> > qemu:///system
> > 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null)
> > (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver: v1,
> > Flags: 11,
> >
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
> > }
> > 2019-08-28 18:59:16,334 DEBUG [kvm.resource.LibvirtComputingResource]
> > (Agent-Handler-4:null) (logid:1f7ef831) Executing:
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > cleanup_rules
> > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
> > file name 'eth1.100'
> > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> > (Agent-Handler-4:null) (logid:1f7ef831) Executing while with timeout :
> > 1800000
> > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
> > file name 'eth1'
> > 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq 1-4941856166108987394:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> >
> [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
> > }
> > 2019-08-28 18:59:16,396 DEBUG [kvm.resource.LibvirtComputingResource]
> > (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
> > 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
> > (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: , MgmtId:
> > 157082893985413, via: 1, Ver: v1, Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
> > 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
> > (logid:) Received response: Seq 0-2: { Ans: , MgmtId: 157082893985413,
> > via: 1, Ver: v1, Flags: 100010,
> >
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
> > }
> > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100011,
> >
> [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
> > }
> > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.ModifyStoragePoolCommand
> > 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create
> storage
> > pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in libvirt
> > 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> > connection at: qemu:///system
> > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing defined
> > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
> > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch storage
> pool
> > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> > connection at: qemu:///system
> > 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed pool
> > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> 727711744
> > Available: 984168267776
> > 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq 1-4941856166108987395:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> >
> [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
> > }
> > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100111,
> >
> [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
> > }
> > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100011, [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> }
> > 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.ModifySshKeysCommand
> > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.SetHostParamsCommand
> > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-1:null) (logid:1f7ef831) Seq 1-4941856166108987397:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100011, [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}]
> }
> > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.ModifySshKeysCommand
> > 2019-08-28 18:59:16,529 DEBUG
> > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600
> > /root/.ssh/id_rsa.cloud
> > 2019-08-28 18:59:16,529 DEBUG
> > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600
> > /root/.ssh/id_rsa.cloud
> > 2019-08-28 18:59:16,531 DEBUG
> > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with
> timeout
> > : 1800000
> > 2019-08-28 18:59:16,532 DEBUG
> > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with
> timeout
> > : 1800000
> > 2019-08-28 18:59:16,534 DEBUG
> > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is successful.
> > 2019-08-28 18:59:16,534 DEBUG
> > [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is successful.
> > 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq 1-4941856166108987398:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-5:null) (logid:1f7ef831) Seq 1-4941856166108987396:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100111,
> > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
> > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.ReadyCommand
> > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent ready
> > command, agent id = 1
> > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
> > 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is processed
> > for agent id = 1
> > 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq 1-4941856166108987399:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> > 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> > Flags: 100111,
> > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
> > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> > com.cloud.agent.api.ReadyCommand
> > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent ready
> > command, agent id = 1
> > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
> > 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is processed
> > for agent id = 1
> > 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq 1-4941856166108987400:
> > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> > 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
> > com.cloud.agent.api.GetHostStatsCommand
> > 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
> > com.cloud.agent.api.GetStorageStatsCommand
> > 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch storage
> pool
> > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd
> > connection at: qemu:///system
> > 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to refresh
> > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> > 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed pool
> > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> 727711744
> > Available: 984168267776
> > 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
> > (Thread-8:null) (logid:) Looking for libvirtd connection at:
> qemu:///system
> > 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> > (logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 in
> > libvirt, continuing
> > 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> > (logid:) Executing:
> > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> > 192.168.0.2 -p /home/primary1 -m
> /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
> > -h 192.168.0.2
> > 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> > (logid:) Executing while with timeout : 60000
> > 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> > (logid:) Execution is successful.
> > 2019-08-28 19:00:16,278 DEBUG [kvm.resource.LibvirtComputingResource]
> > (UgentTask-2:null) (logid:) Executing:
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > get_rule_logs_for_vms
> > 2019-08-28 19:00:16,281 DEBUG [kvm.resource.LibvirtComputingResource]
> > (UgentTask-2:null) (logid:) Executing while with timeout : 1800000
> > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtComputingResource]
> > (UgentTask-2:null) (logid:) Execution is successful.
> > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
> > (UgentTask-2:null) (logid:) Looking for libvirtd connection at:
> > qemu:///system
> > 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null)
> > (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver: v1,
> > Flags: 11,
> >
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
> > }
> > 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
> > (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
> > 157082893985413, via: 1, Ver: v1, Flags: 100010,
> >
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
> > }
> > 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
> > com.cloud.agent.api.GetHostStatsCommand
> > 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:79632973) Processing command:
> > com.cloud.agent.api.GetStorageStatsCommand
> > 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch storage
> pool
> > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> > 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd
> > connection at: qemu:///system
> > 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to refresh
> > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> > 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> > (agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed pool
> > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used:
> 727711744
> > Available: 984168267776
> > [root@fr-kvm1 primary1]#
> >
> > On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> >> I Don't see anything attached from agent.log? Make sure to put agent in
> >> debug mode before posting logs (google for "cloudstack agent debug
> mode",
> >> there is a simple sed command, restart agent afterwards).
> >>
> >> Andrija
> >>
> >> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <md...@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > Also this is agent log
> >> >
> >> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <
> mdvlinquest@gmail.com
> >> >
> >> > wrote:
> >> >
> >> > > Any idea?
> >> > >
> >> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
> >> mdvlinquest@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> Thanks. I changed VLAN ID and now they are kept starting for along
> >> time.
> >> > >> Latest logs:
> >> > >>
> >> > >>
> >> > >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been
> >> pending
> >> > for
> >> > >> 959 seconds
> >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been
> >> pending
> >> > for
> >> > >> 357 seconds
> >> > >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:34:14,852 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> >> routers to
> >> > >> update status.
> >> > >> 2019-08-28 14:34:14,854 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC
> >> > networks
> >> > >> to update Redundant State.
> >> > >> 2019-08-28 14:34:14,857 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
> >> networks
> >> > to
> >> > >> update RvR status.
> >> > >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598229
> >> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598229
> >> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> Resetting
> >> > hosts
> >> > >> suitable for reconnect
> >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> Completed
> >> > >> resetting hosts suitable for reconnect
> >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> Acquiring
> >> > hosts
> >> > >> for clusters already owned by this management server
> >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> Completed
> >> > >> acquiring hosts for clusters already owned by this management
> server
> >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> Acquiring
> >> > hosts
> >> > >> for clusters not owned by any management server
> >> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1)
> Completed
> >> > >> acquiring hosts for clusters not owned by any management server
> >> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> >> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> >> > >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
> >> fr-kvm1.r9host.com)
> >> > >> 2019-08-28 14:34:24,974 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state
> report
> >> > from
> >> > >> ping process. host: 1
> >> > >> 2019-08-28 14:34:24,975 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Process VM state report.
> >> host:
> >> > 1,
> >> > >> number of records in report: 1
> >> > >> 2019-08-28 14:34:24,975 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1,
> vm
> >> id:
> >> > >> 103, power state: PowerOn
> >> > >> 2019-08-28 14:34:24,976 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> >> > host:
> >> > >> 1, vm id: 103, power state: PowerOn
> >> > >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
> >> tasks
> >> > >> working on the VM. vm id: 103, postpone power-change report by
> >> resetting
> >> > >> power-change counters
> >> > >> 2019-08-28 14:34:24,980 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report.
> >> current
> >> > >> time: 1566995664980
> >> > >> 2019-08-28 14:34:24,980 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host:
> >> 1, vm
> >> > >> id: 101, power state: PowerReportMissing, last state update:
> >> > 1566995544000
> >> > >> 2019-08-28 14:34:24,980 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since
> last
> >> > state
> >> > >> update(120980ms) has passed graceful period
> >> > >> 2019-08-28 14:34:24,982 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> >> > host:
> >> > >> 1, vm id: 101, power state: PowerReportMissing
> >> > >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
> >> tasks
> >> > >> working on the VM. vm id: 101, postpone power-change report by
> >> resetting
> >> > >> power-change counters
> >> > >> 2019-08-28 14:34:24,984 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host:
> >> 1, vm
> >> > >> id: 102, power state: PowerReportMissing, last state update:
> >> > 1566995544000
> >> > >> 2019-08-28 14:34:24,984 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since
> last
> >> > state
> >> > >> update(120980ms) has passed graceful period
> >> > >> 2019-08-28 14:34:24,986 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> >> > host:
> >> > >> 1, vm id: 102, power state: PowerReportMissing
> >> > >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
> >> tasks
> >> > >> working on the VM. vm id: 102, postpone power-change report by
> >> resetting
> >> > >> power-change counters
> >> > >> 2019-08-28 14:34:24,987 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Done with process of VM
> state
> >> > >> report. host: 1
> >> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598231
> >> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598231
> >> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> >> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> >> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling
> Monitor
> >> is
> >> > >> running...
> >> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> >> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> >> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> >> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> >> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> >> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
> >> > >> Discovery plugin was unable to find an api by that name or process
> >> any
> >> > apis
> >> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> >> > >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> >> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> >> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector
> >> is
> >> > >> running...
> >> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
> >> > 1-5778962746846479041:
> >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> fr-kvm1.r9host.com
> >> > ),
> >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector
> is
> >> > >> running...
> >> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no
> >> secondary
> >> > >> storage VM for secondary storage host Secondary1
> >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> >> > getCommandHostDelegation:
> >> > >> class com.cloud.agent.api.GetStorageStatsCommand
> >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning
> the
> >> > >> default host to execute commands because the command is not of Copy
> >> > type.
> >> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
> >> > 1-5778962746846479042:
> >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> fr-kvm1.r9host.com
> >> > ),
> >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> >> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> >> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> >> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> >> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> >> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
> >> > >> Discovery plugin was unable to find an api by that name or process
> >> any
> >> > apis
> >> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> >> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598240
> >> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598240
> >> > >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:34:44,852 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> >> routers to
> >> > >> update status.
> >> > >> 2019-08-28 14:34:44,855 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC
> >> > networks
> >> > >> to update Redundant State.
> >> > >> 2019-08-28 14:34:44,857 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
> >> networks
> >> > to
> >> > >> update RvR status.
> >> > >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> >> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:48,215 WARN
> [c.c.a.d.ParamGenericValidationWorker]
> >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> >> Received
> >> > >> unknown parameters for command listSystemVms. Unknown parameters :
> >> > listall
> >> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> >> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listHosts&response=json&details=min&_=1566995598242
> >> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> > >>>Searching
> >> > >> for hosts>>>
> >> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> > >> >>>Generating Response>>>
> >> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listHosts&response=json&details=min&_=1566995598242
> >> > >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> >> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> >> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> >> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> >> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598245
> >> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598245
> >> > >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> >> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> >> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> >> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> >> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> >> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> >> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> >> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
> >> > >> Discovery plugin was unable to find an api by that name or process
> >> any
> >> > apis
> >> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been
> >> pending
> >> > for
> >> > >> 1019 seconds
> >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been
> >> pending
> >> > for
> >> > >> 417 seconds
> >> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> >> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> >> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
> >> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started
> >> resource
> >> > >> counters recalculation periodic task.
> >> > >> 2019-08-28 14:35:14,848 DEBUG
> [c.c.n.ExternalDeviceUsageManagerImpl]
> >> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External
> >> > devices
> >> > >> are not used. Skipping external device usage collection
> >> > >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
> >> > scheduler.poll is
> >> > >> being called at 2019-08-28 12:35:14 GMT
> >> > >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:35:14,857 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> >> routers to
> >> > >> update status.
> >> > >> 2019-08-28 14:35:14,857 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > (RouterMonitor-1:ctx-7cb475c8)
> >> > >> (logid:cc3e421d) Found 0 running routers.
> >> > >> 2019-08-28 14:35:14,858 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC
> >> > networks
> >> > >> to update Redundant State.
> >> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to
> >> be
> >> > >> executed at 2019-08-28 12:35:14 GMT
> >> > >> 2019-08-28 14:35:14,858 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
> >> networks
> >> > to
> >> > >> update RvR status.
> >> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
> >> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
> >> > NetworkGarbageCollector
> >> > >> uses '600' seconds for GC interval.
> >> > >> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
> >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage
> server
> >> > >> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
> >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true,
> >> > heartbeat:
> >> > >> Wed Aug 28 14:35:12 CEST 2019
> >> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> >> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> >> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> >> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> >> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> >> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
> >> > >> Discovery plugin was unable to find an api by that name or process
> >> any
> >> > apis
> >> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> >> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598254
> >> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listInfrastructure&response=json&_=1566995598254
> >> > >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
> >> Manager
> >> > is
> >> > >> running and getting the updates from LB providers and updating
> >> service
> >> > >> status
> >> > >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
> >> Manager
> >> > is
> >> > >> running and getting the updates from LB providers and updating
> >> service
> >> > >> status
> >> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Ping from 1(
> >> fr-kvm1.r9host.com)
> >> > >> 2019-08-28 14:35:25,005 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Process host VM state report
> >> from
> >> > >> ping process. host: 1
> >> > >> 2019-08-28 14:35:25,006 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Process VM state report.
> >> host: 1,
> >> > >> number of records in report: 1
> >> > >> 2019-08-28 14:35:25,006 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm
> >> id:
> >> > >> 103, power state: PowerOn
> >> > >> 2019-08-28 14:35:25,007 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) VM state report is updated.
> >> > host: 1,
> >> > >> vm id: 103, power state: PowerOn
> >> > >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) There is pending job or HA
> >> tasks
> >> > >> working on the VM. vm id: 103, postpone power-change report by
> >> resetting
> >> > >> power-change counters
> >> > >> 2019-08-28 14:35:25,011 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Run missing VM report.
> current
> >> > time:
> >> > >> 1566995725011
> >> > >> 2019-08-28 14:35:25,012 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host:
> 1,
> >> vm
> >> > id:
> >> > >> 101, power state: PowerReportMissing, last state update:
> >> 1566995664000
> >> > >> 2019-08-28 14:35:25,012 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last
> >> > state
> >> > >> update(61011ms) has not passed graceful period yet
> >> > >> 2019-08-28 14:35:25,012 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host:
> 1,
> >> vm
> >> > id:
> >> > >> 102, power state: PowerReportMissing, last state update:
> >> 1566995664000
> >> > >> 2019-08-28 14:35:25,012 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last
> >> > state
> >> > >> update(61011ms) has not passed graceful period yet
> >> > >> 2019-08-28 14:35:25,012 DEBUG
> >> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) Done with process of VM
> state
> >> > >> report. host: 1
> >> > >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> >> > >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:26,390 WARN
> [c.c.a.d.ParamGenericValidationWorker]
> >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> >> Received
> >> > >> unknown parameters for command listSystemVms. Unknown parameters :
> >> > listall
> >> > >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> >> > >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listHosts&response=json&details=min&_=1566995598256
> >> > >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
> >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> >> > >>>Searching
> >> > >> for hosts>>>
> >> > >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
> >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> >> > >> >>>Generating Response>>>
> >> > >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >> command=listHosts&response=json&details=min&_=1566995598256
> >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq
> >> > 1-5778962746846479043:
> >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> fr-kvm1.r9host.com
> >> > ),
> >> > >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
> >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from
> >> executing
> >> > >> class com.cloud.agent.api.GetVolumeStatsCommand:
> >> > >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling
> Monitor
> >> is
> >> > >> running...
> >> > >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> >> > >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> >> > >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> >> > >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> >> > >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector
> >> is
> >> > >> running...
> >> > >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq
> >> > 1-5778962746846479044:
> >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> fr-kvm1.r9host.com
> >> > ),
> >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector
> is
> >> > >> running...
> >> > >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no
> >> secondary
> >> > >> storage VM for secondary storage host Secondary1
> >> > >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
> >> > getCommandHostDelegation:
> >> > >> class com.cloud.agent.api.GetStorageStatsCommand
> >> > >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
> >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning
> the
> >> > >> default host to execute commands because the command is not of Copy
> >> > type.
> >> > >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq
> >> > 1-5778962746846479045:
> >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> >> fr-kvm1.r9host.com
> >> > ),
> >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >> > >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> >> > >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> >> > >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> >> > >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> >> > >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> >> > >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
> >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
> CIDRs
> >> > from
> >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >> > allowed
> >> > >> to perform API calls: 0.0.0.0/0,::/0
> >> > >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
> >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
> >> ===END===
> >> > >> 137.74.35.102 -- GET
> >> > >>
> >> >
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> >> > >> 2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
> >> Checking
> >> > if
> >> > >> any host reservation can be released ...
> >> > >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin
> cleanup
> >> > >> expired async-jobs
> >> > >> 2019-08-28 14:35:44,852 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
> >> routers to
> >> > >> update status.
> >> > >> 2019-08-28 14:35:44,854 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC
> >> > networks
> >> > >> to update Redundant State.
> >> > >> 2019-08-28 14:35:44,856 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
> >> networks
> >> > to
> >> > >> update RvR status.
> >> > >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
> Cannot
> >> > >> release reservation, Found 3 VMs Running on host 1
> >> > >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done
> >> > running
> >> > >> HostReservationReleaseChecker ...
> >> > >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup
> >> > expired
> >> > >> async-jobs
> >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity
> >> Checker
> >> > >> ...
> >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating
> system
> >> > >> capacity
> >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram
> >> > capacity
> >> > >> update
> >> > >
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
Any idea? I have also already managed devpts issue which causes issues on
CentOS 8. I have copied devpts line from /etc/fstab from another host. Also
SELinux is already disabled
On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <md...@gmail.com>
wrote:
> This is agent log at debug log level:
>
> [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
>
> 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
> (logid:) Received response: Seq 1-1: { Ans: , MgmtId: 157082893985413,
> via: -1, Ver: v1, Flags: 100000,
> [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
> }
> 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
> (logid:) Startup task cancelled
> 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> (logid:) Proccess agent startup answer, agent id = 0
> 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> (logid:) Set agent id 0
> 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
> (logid:) Adding a watch list
> 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> (logid:) Startup Response Received: agent id = 0
> 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
> (UgentTask-1:null) (logid:) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> get_rule_logs_for_vms
> 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
> (UgentTask-1:null) (logid:) Executing while with timeout : 1800000
> 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
> }
> 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.CleanupNetworkRulesCmd
> 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
> 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Seq 1-4941856166108987393:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
> (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
>
> jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
> 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
> (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to scan 1
> urls, producing 5 keys and 147 values
> 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> java.lang.Comparable -> java.lang.Enum
> 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
> (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
> java.io.Serializable -> java.lang.Enum
> 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100111,
> [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
> }
> 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.CheckNetworkCommand
> 2019-08-28 18:59:16,327 DEBUG [kvm.resource.LibvirtComputingResource]
> (UgentTask-1:null) (logid:) Execution is successful.
> 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
> (UgentTask-1:null) (logid:) Looking for libvirtd connection at:
> qemu:///system
> 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null)
> (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver: v1,
> Flags: 11,
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
> }
> 2019-08-28 18:59:16,334 DEBUG [kvm.resource.LibvirtComputingResource]
> (Agent-Handler-4:null) (logid:1f7ef831) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> cleanup_rules
> 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
> file name 'eth1.100'
> 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> (Agent-Handler-4:null) (logid:1f7ef831) Executing while with timeout :
> 1800000
> 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
> file name 'eth1'
> 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Seq 1-4941856166108987394:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
> }
> 2019-08-28 18:59:16,396 DEBUG [kvm.resource.LibvirtComputingResource]
> (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
> 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
> (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: , MgmtId:
> 157082893985413, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
> 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
> (logid:) Received response: Seq 0-2: { Ans: , MgmtId: 157082893985413,
> via: 1, Ver: v1, Flags: 100010,
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
> }
> 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
> }
> 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.ModifyStoragePoolCommand
> 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create storage
> pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in libvirt
> 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> connection at: qemu:///system
> 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing defined
> storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
> 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch storage pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
> connection at: qemu:///system
> 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: 727711744
> Available: 984168267776
> 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Seq 1-4941856166108987395:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
> }
> 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100111,
> [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
> }
> 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100011, [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] }
> 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.ModifySshKeysCommand
> 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.SetHostParamsCommand
> 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-1:null) (logid:1f7ef831) Seq 1-4941856166108987397:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100011, [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] }
> 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.ModifySshKeysCommand
> 2019-08-28 18:59:16,529 DEBUG
> [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600
> /root/.ssh/id_rsa.cloud
> 2019-08-28 18:59:16,529 DEBUG
> [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600
> /root/.ssh/id_rsa.cloud
> 2019-08-28 18:59:16,531 DEBUG
> [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with timeout
> : 1800000
> 2019-08-28 18:59:16,532 DEBUG
> [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with timeout
> : 1800000
> 2019-08-28 18:59:16,534 DEBUG
> [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is successful.
> 2019-08-28 18:59:16,534 DEBUG
> [resource.wrapper.LibvirtModifySshKeysCommandWrapper]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is successful.
> 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:1f7ef831) Seq 1-4941856166108987398:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:1f7ef831) Seq 1-4941856166108987396:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100111,
> [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
> 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.ReadyCommand
> 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent ready
> command, agent id = 1
> 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
> 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is processed
> for agent id = 1
> 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:1f7ef831) Seq 1-4941856166108987399:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
> 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
> Flags: 100111,
> [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
> 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
> com.cloud.agent.api.ReadyCommand
> 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent ready
> command, agent id = 1
> 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
> 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is processed
> for agent id = 1
> 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:1f7ef831) Seq 1-4941856166108987400:
> { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
> com.cloud.agent.api.GetHostStatsCommand
> 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
> com.cloud.agent.api.GetStorageStatsCommand
> 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch storage pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd
> connection at: qemu:///system
> 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to refresh
> storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: 727711744
> Available: 984168267776
> 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
> (Thread-8:null) (logid:) Looking for libvirtd connection at: qemu:///system
> 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> (logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 in
> libvirt, continuing
> 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> (logid:) Executing:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> 192.168.0.2 -p /home/primary1 -m /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
> -h 192.168.0.2
> 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> (logid:) Executing while with timeout : 60000
> 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
> (logid:) Execution is successful.
> 2019-08-28 19:00:16,278 DEBUG [kvm.resource.LibvirtComputingResource]
> (UgentTask-2:null) (logid:) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> get_rule_logs_for_vms
> 2019-08-28 19:00:16,281 DEBUG [kvm.resource.LibvirtComputingResource]
> (UgentTask-2:null) (logid:) Executing while with timeout : 1800000
> 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtComputingResource]
> (UgentTask-2:null) (logid:) Execution is successful.
> 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
> (UgentTask-2:null) (logid:) Looking for libvirtd connection at:
> qemu:///system
> 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null)
> (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver: v1,
> Flags: 11,
> [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
> }
> 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
> (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
> 157082893985413, via: 1, Ver: v1, Flags: 100010,
> [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
> }
> 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
> com.cloud.agent.api.GetHostStatsCommand
> 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:79632973) Processing command:
> com.cloud.agent.api.GetStorageStatsCommand
> 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch storage pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
> 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd
> connection at: qemu:///system
> 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to refresh
> storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
> 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed pool
> 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: 727711744
> Available: 984168267776
> [root@fr-kvm1 primary1]#
>
> On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <an...@gmail.com>
> wrote:
>
>> I Don't see anything attached from agent.log? Make sure to put agent in
>> debug mode before posting logs (google for "cloudstack agent debug mode",
>> there is a simple sed command, restart agent afterwards).
>>
>> Andrija
>>
>> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <md...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > Also this is agent log
>> >
>> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <mdvlinquest@gmail.com
>> >
>> > wrote:
>> >
>> > > Any idea?
>> > >
>> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
>> mdvlinquest@gmail.com>
>> > > wrote:
>> > >
>> > >> Thanks. I changed VLAN ID and now they are kept starting for along
>> time.
>> > >> Latest logs:
>> > >>
>> > >>
>> > >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been
>> pending
>> > for
>> > >> 959 seconds
>> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been
>> pending
>> > for
>> > >> 357 seconds
>> > >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:34:14,852 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
>> routers to
>> > >> update status.
>> > >> 2019-08-28 14:34:14,854 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC
>> > networks
>> > >> to update Redundant State.
>> > >> 2019-08-28 14:34:14,857 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0
>> networks
>> > to
>> > >> update RvR status.
>> > >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598229
>> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598229
>> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Resetting
>> > hosts
>> > >> suitable for reconnect
>> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
>> > >> resetting hosts suitable for reconnect
>> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring
>> > hosts
>> > >> for clusters already owned by this management server
>> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
>> > >> acquiring hosts for clusters already owned by this management server
>> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring
>> > hosts
>> > >> for clusters not owned by any management server
>> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
>> > >> acquiring hosts for clusters not owned by any management server
>> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
>> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
>> > >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
>> fr-kvm1.r9host.com)
>> > >> 2019-08-28 14:34:24,974 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state report
>> > from
>> > >> ping process. host: 1
>> > >> 2019-08-28 14:34:24,975 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Process VM state report.
>> host:
>> > 1,
>> > >> number of records in report: 1
>> > >> 2019-08-28 14:34:24,975 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1, vm
>> id:
>> > >> 103, power state: PowerOn
>> > >> 2019-08-28 14:34:24,976 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
>> > host:
>> > >> 1, vm id: 103, power state: PowerOn
>> > >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 103, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:34:24,980 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report.
>> current
>> > >> time: 1566995664980
>> > >> 2019-08-28 14:34:24,980 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host:
>> 1, vm
>> > >> id: 101, power state: PowerReportMissing, last state update:
>> > 1566995544000
>> > >> 2019-08-28 14:34:24,980 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since last
>> > state
>> > >> update(120980ms) has passed graceful period
>> > >> 2019-08-28 14:34:24,982 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
>> > host:
>> > >> 1, vm id: 101, power state: PowerReportMissing
>> > >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 101, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:34:24,984 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host:
>> 1, vm
>> > >> id: 102, power state: PowerReportMissing, last state update:
>> > 1566995544000
>> > >> 2019-08-28 14:34:24,984 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since last
>> > state
>> > >> update(120980ms) has passed graceful period
>> > >> 2019-08-28 14:34:24,986 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
>> > host:
>> > >> 1, vm id: 102, power state: PowerReportMissing
>> > >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 102, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:34:24,987 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Done with process of VM state
>> > >> report. host: 1
>> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598231
>> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598231
>> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
>> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
>> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling Monitor
>> is
>> > >> running...
>> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
>> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
>> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
>> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
>> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
>> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
>> > >> Discovery plugin was unable to find an api by that name or process
>> any
>> > apis
>> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
>> > >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
>> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
>> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector
>> is
>> > >> running...
>> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
>> > 1-5778962746846479041:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector is
>> > >> running...
>> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no
>> secondary
>> > >> storage VM for secondary storage host Secondary1
>> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
>> > getCommandHostDelegation:
>> > >> class com.cloud.agent.api.GetStorageStatsCommand
>> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
>> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning the
>> > >> default host to execute commands because the command is not of Copy
>> > type.
>> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
>> > 1-5778962746846479042:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
>> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
>> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
>> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
>> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
>> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
>> > >> Discovery plugin was unable to find an api by that name or process
>> any
>> > apis
>> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
>> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598240
>> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598240
>> > >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:34:44,852 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
>> routers to
>> > >> update status.
>> > >> 2019-08-28 14:34:44,855 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC
>> > networks
>> > >> to update Redundant State.
>> > >> 2019-08-28 14:34:44,857 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0
>> networks
>> > to
>> > >> update RvR status.
>> > >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
>> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:48,215 WARN [c.c.a.d.ParamGenericValidationWorker]
>> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
>> Received
>> > >> unknown parameters for command listSystemVms. Unknown parameters :
>> > listall
>> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
>> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listHosts&response=json&details=min&_=1566995598242
>> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> > >>>Searching
>> > >> for hosts>>>
>> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> > >> >>>Generating Response>>>
>> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listHosts&response=json&details=min&_=1566995598242
>> > >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
>> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
>> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
>> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
>> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598245
>> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598245
>> > >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
>> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
>> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
>> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
>> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
>> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
>> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
>> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
>> > >> Discovery plugin was unable to find an api by that name or process
>> any
>> > apis
>> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
>> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been
>> pending
>> > for
>> > >> 1019 seconds
>> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been
>> pending
>> > for
>> > >> 417 seconds
>> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
>> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
>> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
>> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started
>> resource
>> > >> counters recalculation periodic task.
>> > >> 2019-08-28 14:35:14,848 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
>> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External
>> > devices
>> > >> are not used. Skipping external device usage collection
>> > >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
>> > scheduler.poll is
>> > >> being called at 2019-08-28 12:35:14 GMT
>> > >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:35:14,857 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
>> routers to
>> > >> update status.
>> > >> 2019-08-28 14:35:14,857 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > (RouterMonitor-1:ctx-7cb475c8)
>> > >> (logid:cc3e421d) Found 0 running routers.
>> > >> 2019-08-28 14:35:14,858 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC
>> > networks
>> > >> to update Redundant State.
>> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to
>> be
>> > >> executed at 2019-08-28 12:35:14 GMT
>> > >> 2019-08-28 14:35:14,858 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0
>> networks
>> > to
>> > >> update RvR status.
>> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
>> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
>> > NetworkGarbageCollector
>> > >> uses '600' seconds for GC interval.
>> > >> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
>> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage server
>> > >> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
>> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true,
>> > heartbeat:
>> > >> Wed Aug 28 14:35:12 CEST 2019
>> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
>> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
>> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
>> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
>> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
>> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
>> > >> Discovery plugin was unable to find an api by that name or process
>> any
>> > apis
>> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
>> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598254
>> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listInfrastructure&response=json&_=1566995598254
>> > >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
>> Manager
>> > is
>> > >> running and getting the updates from LB providers and updating
>> service
>> > >> status
>> > >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck
>> Manager
>> > is
>> > >> running and getting the updates from LB providers and updating
>> service
>> > >> status
>> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Ping from 1(
>> fr-kvm1.r9host.com)
>> > >> 2019-08-28 14:35:25,005 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Process host VM state report
>> from
>> > >> ping process. host: 1
>> > >> 2019-08-28 14:35:25,006 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Process VM state report.
>> host: 1,
>> > >> number of records in report: 1
>> > >> 2019-08-28 14:35:25,006 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm
>> id:
>> > >> 103, power state: PowerOn
>> > >> 2019-08-28 14:35:25,007 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) VM state report is updated.
>> > host: 1,
>> > >> vm id: 103, power state: PowerOn
>> > >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 103, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:35:25,011 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Run missing VM report. current
>> > time:
>> > >> 1566995725011
>> > >> 2019-08-28 14:35:25,012 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1,
>> vm
>> > id:
>> > >> 101, power state: PowerReportMissing, last state update:
>> 1566995664000
>> > >> 2019-08-28 14:35:25,012 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last
>> > state
>> > >> update(61011ms) has not passed graceful period yet
>> > >> 2019-08-28 14:35:25,012 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1,
>> vm
>> > id:
>> > >> 102, power state: PowerReportMissing, last state update:
>> 1566995664000
>> > >> 2019-08-28 14:35:25,012 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last
>> > state
>> > >> update(61011ms) has not passed graceful period yet
>> > >> 2019-08-28 14:35:25,012 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) Done with process of VM state
>> > >> report. host: 1
>> > >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
>> > >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:26,390 WARN [c.c.a.d.ParamGenericValidationWorker]
>> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
>> Received
>> > >> unknown parameters for command listSystemVms. Unknown parameters :
>> > listall
>> > >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
>> > >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
>> > >> 137.74.35.102 -- GET
>> > >> command=listHosts&response=json&details=min&_=1566995598256
>> > >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> > >>>Searching
>> > >> for hosts>>>
>> > >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
>> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> > >> >>>Generating Response>>>
>> > >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >> command=listHosts&response=json&details=min&_=1566995598256
>> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq
>> > 1-5778962746846479043:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
>> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from
>> executing
>> > >> class com.cloud.agent.api.GetVolumeStatsCommand:
>> > >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling Monitor
>> is
>> > >> running...
>> > >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
>> > >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
>> > >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
>> > >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
>> > >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector
>> is
>> > >> running...
>> > >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq
>> > 1-5778962746846479044:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector is
>> > >> running...
>> > >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no
>> secondary
>> > >> storage VM for secondary storage host Secondary1
>> > >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
>> > getCommandHostDelegation:
>> > >> class com.cloud.agent.api.GetStorageStatsCommand
>> > >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
>> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning the
>> > >> default host to execute commands because the command is not of Copy
>> > type.
>> > >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq
>> > 1-5778962746846479045:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> > >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
>> > >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
>> > >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
>> > >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
>> > >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
>> > >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
>> > >> 2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f)
>> Checking
>> > if
>> > >> any host reservation can be released ...
>> > >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:35:44,852 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
>> routers to
>> > >> update status.
>> > >> 2019-08-28 14:35:44,854 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC
>> > networks
>> > >> to update Redundant State.
>> > >> 2019-08-28 14:35:44,856 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0
>> networks
>> > to
>> > >> update RvR status.
>> > >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Cannot
>> > >> release reservation, Found 3 VMs Running on host 1
>> > >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done
>> > running
>> > >> HostReservationReleaseChecker ...
>> > >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity
>> Checker
>> > >> ...
>> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating system
>> > >> capacity
>> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram
>> > capacity
>> > >> update
>> > >> 2019-08-28 14:35:44,896 DEBUG [c.c.c.CapacityManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 3 VMs on host 1
>> > >> 2019-08-28 14:35:44,904 DEBUG [c.c.c.CapacityManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 0 VM, not
>> running
>> > on
>> > >> host 1
>> > >> 2019-08-28 14:35:44,907 DEBUG [c.c.c.CapacityManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
>> cpu
>> > >> capacity, host:1 usedCpuCore: 3 reservedCpuCore: 0
>> > >> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
>> cpu
>> > >> capacity, host:1 usedCpu: 1500 reservedCpu: 0
>> > >> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
>> > memory
>> > >> capacity, host:1 usedMem: 1879048192 reservedMem: 0
>> > >> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing
>> cpu/ram
>> > >> capacity update
>> > >> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing storage
>> > capacity
>> > >> update
>> > >> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found storage pool
>> > Primary1
>> > >> of type SharedMountPoint
>> > >> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Total over
>> provisioned
>> > >> capacity of the pool Primary1 id: 1 is 984895062016
>> > >> 2019-08-28 14:35:44,920 DEBUG [c.c.s.StorageManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Successfully set
>> > Capacity -
>> > >> 984895062016 for capacity type - 3 , DataCenterId - 1, HostOrPoolId
>> - 1,
>> > >> PodId 1
>> > >> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing
>> storage
>> > >> capacity update
>> > >> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity
>> > updates
>> > >> for public ip and Vlans
>> > >> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done capacity updates
>> > for
>> > >> public ip and Vlans
>> > >> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity
>> > updates
>> > >> for private ip
>> > >> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing
>> capacity
>> > >> updates for private ip
>> > >> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done recalculating
>> > system
>> > >> capacity
>> > >> 2019-08-28 14:35:44,982 DEBUG [c.c.a.AlertManagerImpl]
>> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done running Capacity
>> > >> Checker ...
>> > >> 2019-08-28 14:35:49,922 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Resetting
>> > hosts
>> > >> suitable for reconnect
>> > >> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
>> > >> resetting hosts suitable for reconnect
>> > >> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring
>> > hosts
>> > >> for clusters already owned by this management server
>> > >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
>> > >> acquiring hosts for clusters already owned by this management server
>> > >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring
>> > hosts
>> > >> for clusters not owned by any management server
>> > >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
>> > >> acquiring hosts for clusters not owned by any management server
>> > >> 2019-08-28 14:35:49,953 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-5e476ef2) (logid:be235fb1) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
>> > >> 2019-08-28 14:35:49,961 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:50,014 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
>> > >> 2019-08-28 14:35:50,633 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-3f36683f) (logid:7cc15823) ===START===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
>> > >> 2019-08-28 14:35:50,643 DEBUG [c.c.a.ApiServer]
>> > >> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) CIDRs
>> > from
>> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > allowed
>> > >> to perform API calls: 0.0.0.0/0,::/0
>> > >> 2019-08-28 14:35:50,698 DEBUG [c.c.a.ApiServlet]
>> > >> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823)
>> ===END===
>> > >> 137.74.35.102 -- GET
>> > >>
>> >
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
>> > >> 2019-08-28 14:35:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:35:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:36:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:36:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-318) has been
>> pending
>> > for
>> > >> 1079 seconds
>> > >> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> > >> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-320) has been
>> pending
>> > for
>> > >> 477 seconds
>> > >> 2019-08-28 14:36:14,855 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:36:14,855 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0
>> routers to
>> > >> update status.
>> > >> 2019-08-28 14:36:14,857 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 VPC
>> > networks
>> > >> to update Redundant State.
>> > >> 2019-08-28 14:36:14,859 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0
>> networks
>> > to
>> > >> update RvR status.
>> > >> 2019-08-28 14:36:14,864 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:36:24,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:36:24,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:36:24,972 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Ping from 1(
>> fr-kvm1.r9host.com)
>> > >> 2019-08-28 14:36:24,972 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Process host VM state report
>> from
>> > >> ping process. host: 1
>> > >> 2019-08-28 14:36:24,973 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Process VM state report.
>> host: 1,
>> > >> number of records in report: 1
>> > >> 2019-08-28 14:36:24,973 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) VM state report. host: 1, vm
>> id:
>> > >> 103, power state: PowerOn
>> > >> 2019-08-28 14:36:24,975 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
>> > host: 1,
>> > >> vm id: 103, power state: PowerOn
>> > >> 2019-08-28 14:36:24,976 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 103, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:36:24,978 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Run missing VM report. current
>> > time:
>> > >> 1566995784978
>> > >> 2019-08-28 14:36:24,979 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1,
>> vm
>> > id:
>> > >> 101, power state: PowerReportMissing, last state update:
>> 1566995664000
>> > >> 2019-08-28 14:36:24,979 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) vm id: 101 - time since last
>> > state
>> > >> update(120978ms) has passed graceful period
>> > >> 2019-08-28 14:36:24,981 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
>> > host: 1,
>> > >> vm id: 101, power state: PowerReportMissing
>> > >> 2019-08-28 14:36:24,981 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 101, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:36:24,983 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1,
>> vm
>> > id:
>> > >> 102, power state: PowerReportMissing, last state update:
>> 1566995664000
>> > >> 2019-08-28 14:36:24,983 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) vm id: 102 - time since last
>> > state
>> > >> update(120978ms) has passed graceful period
>> > >> 2019-08-28 14:36:24,984 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
>> > host: 1,
>> > >> vm id: 102, power state: PowerReportMissing
>> > >> 2019-08-28 14:36:24,984 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA
>> tasks
>> > >> working on the VM. vm id: 102, postpone power-change report by
>> resetting
>> > >> power-change counters
>> > >> 2019-08-28 14:36:24,986 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) Done with process of VM state
>> > >> report. host: 1
>> > >> 2019-08-28 14:36:30,181 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-6:ctx-9c8e3bed) (logid:9058cad8) AutoScaling Monitor
>> is
>> > >> running...
>> > >> 2019-08-28 14:36:34,848 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) Begin cleanup
>> > >> expired async-jobs
>> > >> 2019-08-28 14:36:34,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) End cleanup
>> > expired
>> > >> async-jobs
>> > >> 2019-08-28 14:36:37,458 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) HostStatsCollector
>> is
>> > >> running...
>> > >> 2019-08-28 14:36:37,515 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) Seq
>> > 1-5778962746846479046:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2019-08-28 14:36:38,661 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) StorageCollector is
>> > >> running...
>> > >> 2019-08-28 14:36:38,671 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) There is no
>> secondary
>> > >> storage VM for secondary storage host Secondary1
>> > >> 2019-08-28 14:36:38,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca)
>> > getCommandHostDelegation:
>> > >> class com.cloud.agent.api.GetStorageStatsCommand
>> > >> 2019-08-28 14:36:38,675 DEBUG [c.c.h.XenServerGuru]
>> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) We are returning the
>> > >> default host to execute commands because the command is not of Copy
>> > type.
>> > >> 2019-08-28 14:36:38,729 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) Seq
>> > 1-5778962746846479047:
>> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> > >>
>> > >> On Wed, Aug 28, 2019 at 4:04 PM Paul Angus <paul.angus@shapeblue.com
>> >
>> > >> wrote:
>> > >>
>> > >>> The error is here Fariborz,
>> > >>>
>> > >>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
>> > >>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> > >>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to
>> Failed
>> > to
>> > >>> create vnet 100: device eth1.100 is already a member of a bridge;
>> can't
>> > >>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>> > >>> breth1-100
>> > >>>
>> > >>> you're already using a VLAN that you've also asked CloudStack to
>> use.
>> > >>>
>> > >>>
>> > >>>
>> > >>> paul.angus@shapeblue.com
>> > >>> www.shapeblue.com
>> > >>> Amadeus House, Floral Street, London WC2E 9DPUK
>> > >>> @shapeblue
>> > >>>
>> > >>>
>> > >>>
>> > >>>
>> > >>> -----Original Message-----
>> > >>> From: Fariborz Navidan <md...@gmail.com>
>> > >>> Sent: 28 August 2019 12:29
>> > >>> To: users@cloudstack.apache.org
>> > >>> Subject: Unable to start System VMs
>> > >>>
>> > >>> Hello,
>> > >>>
>> > >>> New ACS management is unable to start System VMs. Below are logs.
>> ACS
>> > is
>> > >>> still poor in error handling and reporting. It should be able to
>> retyrn
>> > >>> clear error messages inside UI.
>> > >>>
>> > >>> 2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin
>> cleanup
>> > >>> expired async-jobs
>> > >>> 2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup
>> > >>> expired
>> > >>> async-jobs
>> > >>> 2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin
>> cleanup
>> > >>> expired async-jobs
>> > >>> 2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup
>> > >>> expired
>> > >>> async-jobs
>> > >>> 2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
>> > >>> 2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170)
>> > ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
>> > >>> 2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START===
>> > >>> 137.74.35.102
>> > >>> -- GET
>> > >>>
>> > >>>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
>> > >>> 2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5)
>> ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
>> > >>> 2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
>> > >>> 2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933)
>> > ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
>> > >>> 2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START===
>> > >>> 137.74.35.102
>> > >>> -- GET
>> > command=listApis&response=json&name=listVmwareDcs&_=1566988128967
>> > >>> 2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api
>> > >>> Discovery
>> > >>> plugin was unable to find an api by that name or process any apis
>> > >>> 2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719)
>> ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
>> > >>> 2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin
>> cleanup
>> > >>> expired async-jobs
>> > >>> 2019-08-28 13:22:14,852 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0
>> routers
>> > to
>> > >>> update status.
>> > >>> 2019-08-28 13:22:14,854 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC
>> > >>> networks
>> > >>> to update Redundant State.
>> > >>> 2019-08-28 13:22:14,856 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0
>> networks
>> > to
>> > >>> update RvR status.
>> > >>> 2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup
>> > >>> expired
>> > >>> async-jobs
>> > >>> 2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
>> > >>> 2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8)
>> > ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
>> > >>> 2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START===
>> > >>> 137.74.35.102
>> > >>> -- GET
>> > >>>
>> > >>>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
>> > >>> 2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41)
>> ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
>> > >>> 2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
>> > >>> 2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05)
>> > ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>>
>> > >>>
>> >
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
>> > >>> 2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START===
>> > >>> 137.74.35.102
>> > >>> -- GET
>> > command=listApis&response=json&name=listVmwareDcs&_=1566988128971
>> > >>> 2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api
>> > >>> Discovery
>> > >>> plugin was unable to find an api by that name or process any apis
>> > >>> 2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5)
>> ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
>> > >>> 2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl]
>> > >>> (ClusteredAgentManager
>> > >>> Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for
>> > >>> reconnect
>> > >>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
>> > >>> (ClusteredAgentManager
>> > >>> Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts
>> suitable
>> > >>> for
>> > >>> reconnect
>> > >>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
>> > >>> (ClusteredAgentManager
>> > >>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters
>> > already
>> > >>> owned by this management server
>> > >>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
>> > >>> (ClusteredAgentManager
>> > >>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
>> > >>> clusters
>> > >>> already owned by this management server
>> > >>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
>> > >>> (ClusteredAgentManager
>> > >>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters
>> not
>> > >>> owned
>> > >>> by any management server
>> > >>> 2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl]
>> > >>> (ClusteredAgentManager
>> > >>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
>> > >>> clusters
>> > >>> not owned by any management server
>> > >>> 2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listInfrastructure&response=json&_=1566988128972
>> > >>> 2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef)
>> > ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listInfrastructure&response=json&_=1566988128972
>> > >>> 2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin
>> cleanup
>> > >>> expired async-jobs
>> > >>> 2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup
>> > >>> expired
>> > >>> async-jobs
>> > >>> 2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>> (AgentManager-Handler-3:null) (logid:) Ping from 1(
>> fr-kvm1.r9host.com)
>> > >>> 2019-08-28 13:22:24,957 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >>> (AgentManager-Handler-3:null) (logid:) Process host VM state report
>> > from
>> > >>> ping process. host: 1
>> > >>> 2019-08-28 13:22:24,957 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >>> (AgentManager-Handler-3:null) (logid:) Process VM state report.
>> host:
>> > 1,
>> > >>> number of records in report: 0
>> > >>> 2019-08-28 13:22:24,957 DEBUG
>> [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
>> > >>> report. host: 1
>> > >>> 2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
>> > >>> (StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling
>> Monitor is
>> > >>> running...
>> > >>> 2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START===
>> > >>> 137.74.35.102
>> > >>> -- GET command=listInfrastructure&response=json&_=1566988128973
>> > >>> 2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
>> > >>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs
>> > from
>> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> > >>> allowed
>> > >>> to perform API calls: 0.0.0.0/0,::/0
>> > >>> 2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
>> > >>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0)
>> ===END===
>> > >>> 137.74.35.102 -- GET
>> > >>> command=listInfrastructure&response=json&_=1566988128973
>> > >>> 2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
>> > >>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector
>> is
>> > >>> running...
>> > >>> 2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
>> > >>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq
>> > >>> 1-5778962746846478713:
>> > >>> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >>> 2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
>> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
>> > >>> running...
>> > >>> 2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
>> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no
>> secondary
>> > >>> storage VM for secondary storage host Secondary1
>> > >>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20)
>> > >>> getCommandHostDelegation:
>> > >>> class com.cloud.agent.api.GetStorageStatsCommand
>> > >>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
>> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning
>> the
>> > >>> default host to execute commands because the command is not of Copy
>> > type.
>> > >>> 2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
>> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq
>> > >>> 1-5778962746846478714:
>> > >>> Received: { Ans: , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com
>> > ),
>> > >>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> > >>> 2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin
>> cleanup
>> > >>> expired async-jobs
>> > >>> 2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup
>> > >>> expired
>> > >>> async-jobs
>> > >>> 2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to
>> launch
>> > >>> secondary storage VM
>> > >>> 2019-08-28 13:22:40,562 INFO
>> > >>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary
>> > storage
>> > >>> vms found in datacenter id=1, starting one
>> > >>> 2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary
>> > storage
>> > >>> vm is available, need to allocate a new secondary storage vm
>> > >>> 2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary
>> storage
>> > vm
>> > >>> from a newly started instance for request from data center : 1
>> > >>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>> > >>> configuration for offering [Network Offering
>> > >>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]:
>> Ntwk[203|Guest|6]
>> > >>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>> > >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> > >>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>> > >>> configuration for offering [Network Offering
>> > >>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
>> > >>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>> > >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> > >>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>> > >>> configuration for offering [Network Offering
>> > >>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
>> > >>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>> > >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> > >>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to
>> > launch
>> >
>>
>
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
This is agent log at debug log level:
[root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log
2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
(logid:) Received response: Seq 1-1: { Ans: , MgmtId: 157082893985413,
via: -1, Ver: v1, Flags: 100000,
[{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}]
}
2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
(logid:) Startup task cancelled
2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
(logid:) Proccess agent startup answer, agent id = 0
2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
(logid:) Set agent id 0
2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent] (Agent-Handler-2:null)
(logid:) Adding a watch list
2019-08-28 18:59:16,277 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
(logid:) Startup Response Received: agent id = 0
2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-1:null) (logid:) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
get_rule_logs_for_vms
2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-1:null) (logid:) Executing while with timeout : 1800000
2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100011,
[{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}]
}
2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.CleanupNetworkRulesCmd
2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list
2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Seq 1-4941856166108987393:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections]
(Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls:
jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/
2019-08-28 18:59:16,295 INFO [org.reflections.Reflections]
(Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to scan 1
urls, producing 5 keys and 147 values
2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
(Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
java.lang.Comparable -> java.lang.Enum
2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections]
(Agent-Handler-4:null) (logid:1f7ef831) expanded subtype
java.io.Serializable -> java.lang.Enum
2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}]
}
2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.CheckNetworkCommand
2019-08-28 18:59:16,327 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-1:null) (logid:) Execution is successful.
2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection]
(UgentTask-1:null) (logid:) Looking for libvirtd connection at:
qemu:///system
2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null)
(logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver: v1,
Flags: 11,
[{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}]
}
2019-08-28 18:59:16,334 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-4:null) (logid:1f7ef831) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
cleanup_rules
2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
file name 'eth1.100'
2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-4:null) (logid:1f7ef831) Executing while with timeout :
1800000
2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory:
file name 'eth1'
2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Seq 1-4941856166108987394:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}]
}
2019-08-28 18:59:16,396 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-4:null) (logid:1f7ef831) Execution is successful.
2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
(logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: , MgmtId:
157082893985413, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
(logid:) Received response: Seq 0-2: { Ans: , MgmtId: 157082893985413,
via: 1, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}]
}
2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}]
}
2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.ModifyStoragePoolCommand
2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create storage
pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in libvirt
2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
connection at: qemu:///system
2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:1f7ef831) Found existing defined
storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it.
2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch storage pool
5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd
connection at: qemu:///system
2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed pool
5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: 727711744
Available: 984168267776
2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Seq 1-4941856166108987395:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}]
}
2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq
1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}]
}
2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq
1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100011, [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] }
2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.ModifySshKeysCommand
2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.SetHostParamsCommand
2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) (logid:1f7ef831) Seq 1-4941856166108987397:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq
1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100011, [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] }
2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.ModifySshKeysCommand
2019-08-28 18:59:16,529 DEBUG
[resource.wrapper.LibvirtModifySshKeysCommandWrapper]
(agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600
/root/.ssh/id_rsa.cloud
2019-08-28 18:59:16,529 DEBUG
[resource.wrapper.LibvirtModifySshKeysCommandWrapper]
(agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600
/root/.ssh/id_rsa.cloud
2019-08-28 18:59:16,531 DEBUG
[resource.wrapper.LibvirtModifySshKeysCommandWrapper]
(agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with timeout
: 1800000
2019-08-28 18:59:16,532 DEBUG
[resource.wrapper.LibvirtModifySshKeysCommandWrapper]
(agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with timeout
: 1800000
2019-08-28 18:59:16,534 DEBUG
[resource.wrapper.LibvirtModifySshKeysCommandWrapper]
(agentRequest-Handler-5:null) (logid:1f7ef831) Execution is successful.
2019-08-28 18:59:16,534 DEBUG
[resource.wrapper.LibvirtModifySshKeysCommandWrapper]
(agentRequest-Handler-2:null) (logid:1f7ef831) Execution is successful.
2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:1f7ef831) Seq 1-4941856166108987398:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) (logid:1f7ef831) Seq 1-4941856166108987396:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq
1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.ReadyCommand
2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent ready
command, agent id = 1
2019-08-28 18:59:16,542 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1
2019-08-28 18:59:16,551 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is processed
for agent id = 1
2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:1f7ef831) Seq 1-4941856166108987399:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq
1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] }
2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Processing command:
com.cloud.agent.api.ReadyCommand
2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent ready
command, agent id = 1
2019-08-28 18:59:16,641 INFO [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1
2019-08-28 18:59:16,642 INFO [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is processed
for agent id = 1
2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:1f7ef831) Seq 1-4941856166108987400:
{ Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) (logid:9391b96b) Processing command:
com.cloud.agent.api.GetHostStatsCommand
2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command:
com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch storage pool
5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd
connection at: qemu:///system
2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to refresh
storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed pool
5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: 727711744
Available: 984168267776
2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection]
(Thread-8:null) (logid:) Looking for libvirtd connection at: qemu:///system
2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
(logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 in
libvirt, continuing
2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
(logid:) Executing:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
192.168.0.2 -p /home/primary1 -m /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88
-h 192.168.0.2
2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
(logid:) Executing while with timeout : 60000
2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor] (Thread-8:null)
(logid:) Execution is successful.
2019-08-28 19:00:16,278 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-2:null) (logid:) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
get_rule_logs_for_vms
2019-08-28 19:00:16,281 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-2:null) (logid:) Executing while with timeout : 1800000
2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-2:null) (logid:) Execution is successful.
2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection]
(UgentTask-2:null) (logid:) Looking for libvirtd connection at:
qemu:///system
2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null)
(logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver: v1,
Flags: 11,
[{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
}
2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
(logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId:
157082893985413, via: 1, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
}
2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) (logid:919a0a37) Processing command:
com.cloud.agent.api.GetHostStatsCommand
2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:null) (logid:79632973) Processing command:
com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:null) (logid:79632973) Trying to fetch storage pool
5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt
2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd
connection at: qemu:///system
2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to refresh
storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88
2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed pool
5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: 727711744
Available: 984168267776
[root@fr-kvm1 primary1]#
On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <an...@gmail.com>
wrote:
> I Don't see anything attached from agent.log? Make sure to put agent in
> debug mode before posting logs (google for "cloudstack agent debug mode",
> there is a simple sed command, restart agent afterwards).
>
> Andrija
>
> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Also this is agent log
> >
> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> > > Any idea?
> > >
> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <
> mdvlinquest@gmail.com>
> > > wrote:
> > >
> > >> Thanks. I changed VLAN ID and now they are kept starting for along
> time.
> > >> Latest logs:
> > >>
> > >>
> > >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been
> pending
> > for
> > >> 959 seconds
> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been
> pending
> > for
> > >> 357 seconds
> > >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:34:14,852 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 routers
> to
> > >> update status.
> > >> 2019-08-28 14:34:14,854 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC
> > networks
> > >> to update Redundant State.
> > >> 2019-08-28 14:34:14,857 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 networks
> > to
> > >> update RvR status.
> > >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598229
> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598229
> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Resetting
> > hosts
> > >> suitable for reconnect
> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
> > >> resetting hosts suitable for reconnect
> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring
> > hosts
> > >> for clusters already owned by this management server
> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
> > >> acquiring hosts for clusters already owned by this management server
> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring
> > hosts
> > >> for clusters not owned by any management server
> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
> > >> acquiring hosts for clusters not owned by any management server
> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> > >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1(
> fr-kvm1.r9host.com)
> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state report
> > from
> > >> ping process. host: 1
> > >> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Process VM state report. host:
> > 1,
> > >> number of records in report: 1
> > >> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1, vm
> id:
> > >> 103, power state: PowerOn
> > >> 2019-08-28 14:34:24,976 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> > host:
> > >> 1, vm id: 103, power state: PowerOn
> > >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 103, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report. current
> > >> time: 1566995664980
> > >> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1,
> vm
> > >> id: 101, power state: PowerReportMissing, last state update:
> > 1566995544000
> > >> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since last
> > state
> > >> update(120980ms) has passed graceful period
> > >> 2019-08-28 14:34:24,982 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> > host:
> > >> 1, vm id: 101, power state: PowerReportMissing
> > >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 101, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1,
> vm
> > >> id: 102, power state: PowerReportMissing, last state update:
> > 1566995544000
> > >> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since last
> > state
> > >> update(120980ms) has passed graceful period
> > >> 2019-08-28 14:34:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> > host:
> > >> 1, vm id: 102, power state: PowerReportMissing
> > >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 102, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:34:24,987 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Done with process of VM state
> > >> report. host: 1
> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598231
> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598231
> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling Monitor
> is
> > >> running...
> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
> > >> Discovery plugin was unable to find an api by that name or process any
> > apis
> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> > >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector is
> > >> running...
> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
> > 1-5778962746846479041:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector is
> > >> running...
> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no secondary
> > >> storage VM for secondary storage host Secondary1
> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> > getCommandHostDelegation:
> > >> class com.cloud.agent.api.GetStorageStatsCommand
> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning the
> > >> default host to execute commands because the command is not of Copy
> > type.
> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
> > 1-5778962746846479042:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
> > >> Discovery plugin was unable to find an api by that name or process any
> > apis
> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598240
> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598240
> > >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:34:44,852 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 routers
> to
> > >> update status.
> > >> 2019-08-28 14:34:44,855 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC
> > networks
> > >> to update Redundant State.
> > >> 2019-08-28 14:34:44,857 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 networks
> > to
> > >> update RvR status.
> > >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:48,215 WARN [c.c.a.d.ParamGenericValidationWorker]
> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> Received
> > >> unknown parameters for command listSystemVms. Unknown parameters :
> > listall
> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listHosts&response=json&details=min&_=1566995598242
> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> > >>>Searching
> > >> for hosts>>>
> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> > >> >>>Generating Response>>>
> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listHosts&response=json&details=min&_=1566995598242
> > >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598245
> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598245
> > >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
> > >> Discovery plugin was unable to find an api by that name or process any
> > apis
> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been
> pending
> > for
> > >> 1019 seconds
> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been
> pending
> > for
> > >> 417 seconds
> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started
> resource
> > >> counters recalculation periodic task.
> > >> 2019-08-28 14:35:14,848 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External
> > devices
> > >> are not used. Skipping external device usage collection
> > >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
> > scheduler.poll is
> > >> being called at 2019-08-28 12:35:14 GMT
> > >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:35:14,857 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 routers
> to
> > >> update status.
> > >> 2019-08-28 14:35:14,857 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > (RouterMonitor-1:ctx-7cb475c8)
> > >> (logid:cc3e421d) Found 0 running routers.
> > >> 2019-08-28 14:35:14,858 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC
> > networks
> > >> to update Redundant State.
> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to be
> > >> executed at 2019-08-28 12:35:14 GMT
> > >> 2019-08-28 14:35:14,858 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 networks
> > to
> > >> update RvR status.
> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
> > NetworkGarbageCollector
> > >> uses '600' seconds for GC interval.
> > >> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage server
> > >> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true,
> > heartbeat:
> > >> Wed Aug 28 14:35:12 CEST 2019
> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
> > >> Discovery plugin was unable to find an api by that name or process any
> > apis
> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598254
> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listInfrastructure&response=json&_=1566995598254
> > >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager
> > is
> > >> running and getting the updates from LB providers and updating service
> > >> status
> > >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager
> > is
> > >> running and getting the updates from LB providers and updating service
> > >> status
> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Ping from 1(fr-kvm1.r9host.com
> )
> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Process host VM state report
> from
> > >> ping process. host: 1
> > >> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Process VM state report. host:
> 1,
> > >> number of records in report: 1
> > >> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm
> id:
> > >> 103, power state: PowerOn
> > >> 2019-08-28 14:35:25,007 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) VM state report is updated.
> > host: 1,
> > >> vm id: 103, power state: PowerOn
> > >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-1:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 103, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:35:25,011 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Run missing VM report. current
> > time:
> > >> 1566995725011
> > >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1,
> vm
> > id:
> > >> 101, power state: PowerReportMissing, last state update: 1566995664000
> > >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last
> > state
> > >> update(61011ms) has not passed graceful period yet
> > >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1,
> vm
> > id:
> > >> 102, power state: PowerReportMissing, last state update: 1566995664000
> > >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last
> > state
> > >> update(61011ms) has not passed graceful period yet
> > >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-1:null) (logid:) Done with process of VM state
> > >> report. host: 1
> > >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> > >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:26,390 WARN [c.c.a.d.ParamGenericValidationWorker]
> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> Received
> > >> unknown parameters for command listSystemVms. Unknown parameters :
> > listall
> > >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> > >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
> > >> 137.74.35.102 -- GET
> > >> command=listHosts&response=json&details=min&_=1566995598256
> > >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> > >>>Searching
> > >> for hosts>>>
> > >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> > >> >>>Generating Response>>>
> > >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> ===END===
> > >> 137.74.35.102 -- GET
> > >> command=listHosts&response=json&details=min&_=1566995598256
> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq
> > 1-5778962746846479043:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from
> executing
> > >> class com.cloud.agent.api.GetVolumeStatsCommand:
> > >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling Monitor
> is
> > >> running...
> > >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> > >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> > >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> > >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> > >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector is
> > >> running...
> > >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq
> > 1-5778962746846479044:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector is
> > >> running...
> > >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no secondary
> > >> storage VM for secondary storage host Secondary1
> > >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
> > getCommandHostDelegation:
> > >> class com.cloud.agent.api.GetStorageStatsCommand
> > >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning the
> > >> default host to execute commands because the command is not of Copy
> > type.
> > >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq
> > 1-5778962746846479045:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> > >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> > >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> > >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> > >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> > >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> > >> 2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Checking
> > if
> > >> any host reservation can be released ...
> > >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:35:44,852 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 routers
> to
> > >> update status.
> > >> 2019-08-28 14:35:44,854 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC
> > networks
> > >> to update Redundant State.
> > >> 2019-08-28 14:35:44,856 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 networks
> > to
> > >> update RvR status.
> > >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Cannot
> > >> release reservation, Found 3 VMs Running on host 1
> > >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done
> > running
> > >> HostReservationReleaseChecker ...
> > >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity
> Checker
> > >> ...
> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating system
> > >> capacity
> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram
> > capacity
> > >> update
> > >> 2019-08-28 14:35:44,896 DEBUG [c.c.c.CapacityManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 3 VMs on host 1
> > >> 2019-08-28 14:35:44,904 DEBUG [c.c.c.CapacityManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 0 VM, not
> running
> > on
> > >> host 1
> > >> 2019-08-28 14:35:44,907 DEBUG [c.c.c.CapacityManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
> cpu
> > >> capacity, host:1 usedCpuCore: 3 reservedCpuCore: 0
> > >> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
> cpu
> > >> capacity, host:1 usedCpu: 1500 reservedCpu: 0
> > >> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
> > memory
> > >> capacity, host:1 usedMem: 1879048192 reservedMem: 0
> > >> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing cpu/ram
> > >> capacity update
> > >> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing storage
> > capacity
> > >> update
> > >> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found storage pool
> > Primary1
> > >> of type SharedMountPoint
> > >> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Total over provisioned
> > >> capacity of the pool Primary1 id: 1 is 984895062016
> > >> 2019-08-28 14:35:44,920 DEBUG [c.c.s.StorageManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Successfully set
> > Capacity -
> > >> 984895062016 for capacity type - 3 , DataCenterId - 1, HostOrPoolId -
> 1,
> > >> PodId 1
> > >> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing storage
> > >> capacity update
> > >> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity
> > updates
> > >> for public ip and Vlans
> > >> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done capacity updates
> > for
> > >> public ip and Vlans
> > >> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity
> > updates
> > >> for private ip
> > >> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing
> capacity
> > >> updates for private ip
> > >> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done recalculating
> > system
> > >> capacity
> > >> 2019-08-28 14:35:44,982 DEBUG [c.c.a.AlertManagerImpl]
> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done running Capacity
> > >> Checker ...
> > >> 2019-08-28 14:35:49,922 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Resetting
> > hosts
> > >> suitable for reconnect
> > >> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
> > >> resetting hosts suitable for reconnect
> > >> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring
> > hosts
> > >> for clusters already owned by this management server
> > >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
> > >> acquiring hosts for clusters already owned by this management server
> > >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring
> > hosts
> > >> for clusters not owned by any management server
> > >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
> > >> acquiring hosts for clusters not owned by any management server
> > >> 2019-08-28 14:35:49,953 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-5e476ef2) (logid:be235fb1) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
> > >> 2019-08-28 14:35:49,961 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:50,014 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
> > >> 2019-08-28 14:35:50,633 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-3f36683f) (logid:7cc15823) ===START===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
> > >> 2019-08-28 14:35:50,643 DEBUG [c.c.a.ApiServer]
> > >> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) CIDRs
> > from
> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > allowed
> > >> to perform API calls: 0.0.0.0/0,::/0
> > >> 2019-08-28 14:35:50,698 DEBUG [c.c.a.ApiServlet]
> > >> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823)
> ===END===
> > >> 137.74.35.102 -- GET
> > >>
> >
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
> > >> 2019-08-28 14:35:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:35:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:36:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:36:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-318) has been
> pending
> > for
> > >> 1079 seconds
> > >> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > >> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-320) has been
> pending
> > for
> > >> 477 seconds
> > >> 2019-08-28 14:36:14,855 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:36:14,855 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 routers
> to
> > >> update status.
> > >> 2019-08-28 14:36:14,857 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 VPC
> > networks
> > >> to update Redundant State.
> > >> 2019-08-28 14:36:14,859 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 networks
> > to
> > >> update RvR status.
> > >> 2019-08-28 14:36:14,864 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:36:24,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:36:24,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:36:24,972 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Ping from 1(fr-kvm1.r9host.com
> )
> > >> 2019-08-28 14:36:24,972 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Process host VM state report
> from
> > >> ping process. host: 1
> > >> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Process VM state report. host:
> 1,
> > >> number of records in report: 1
> > >> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) VM state report. host: 1, vm
> id:
> > >> 103, power state: PowerOn
> > >> 2019-08-28 14:36:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
> > host: 1,
> > >> vm id: 103, power state: PowerOn
> > >> 2019-08-28 14:36:24,976 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 103, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:36:24,978 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Run missing VM report. current
> > time:
> > >> 1566995784978
> > >> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1,
> vm
> > id:
> > >> 101, power state: PowerReportMissing, last state update: 1566995664000
> > >> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) vm id: 101 - time since last
> > state
> > >> update(120978ms) has passed graceful period
> > >> 2019-08-28 14:36:24,981 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
> > host: 1,
> > >> vm id: 101, power state: PowerReportMissing
> > >> 2019-08-28 14:36:24,981 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 101, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1,
> vm
> > id:
> > >> 102, power state: PowerReportMissing, last state update: 1566995664000
> > >> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) vm id: 102 - time since last
> > state
> > >> update(120978ms) has passed graceful period
> > >> 2019-08-28 14:36:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
> > host: 1,
> > >> vm id: 102, power state: PowerReportMissing
> > >> 2019-08-28 14:36:24,984 INFO [c.c.v.VirtualMachineManagerImpl]
> > >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA
> tasks
> > >> working on the VM. vm id: 102, postpone power-change report by
> resetting
> > >> power-change counters
> > >> 2019-08-28 14:36:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-7:null) (logid:) Done with process of VM state
> > >> report. host: 1
> > >> 2019-08-28 14:36:30,181 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-6:ctx-9c8e3bed) (logid:9058cad8) AutoScaling Monitor
> is
> > >> running...
> > >> 2019-08-28 14:36:34,848 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) Begin cleanup
> > >> expired async-jobs
> > >> 2019-08-28 14:36:34,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) End cleanup
> > expired
> > >> async-jobs
> > >> 2019-08-28 14:36:37,458 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) HostStatsCollector is
> > >> running...
> > >> 2019-08-28 14:36:37,515 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) Seq
> > 1-5778962746846479046:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> 2019-08-28 14:36:38,661 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) StorageCollector is
> > >> running...
> > >> 2019-08-28 14:36:38,671 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) There is no secondary
> > >> storage VM for secondary storage host Secondary1
> > >> 2019-08-28 14:36:38,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca)
> > getCommandHostDelegation:
> > >> class com.cloud.agent.api.GetStorageStatsCommand
> > >> 2019-08-28 14:36:38,675 DEBUG [c.c.h.XenServerGuru]
> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) We are returning the
> > >> default host to execute commands because the command is not of Copy
> > type.
> > >> 2019-08-28 14:36:38,729 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) Seq
> > 1-5778962746846479047:
> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >>
> > >> On Wed, Aug 28, 2019 at 4:04 PM Paul Angus <pa...@shapeblue.com>
> > >> wrote:
> > >>
> > >>> The error is here Fariborz,
> > >>>
> > >>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
> > >>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> > >>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed
> > to
> > >>> create vnet 100: device eth1.100 is already a member of a bridge;
> can't
> > >>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
> > >>> breth1-100
> > >>>
> > >>> you're already using a VLAN that you've also asked CloudStack to use.
> > >>>
> > >>>
> > >>>
> > >>> paul.angus@shapeblue.com
> > >>> www.shapeblue.com
> > >>> Amadeus House, Floral Street, London WC2E 9DPUK
> > >>> @shapeblue
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> -----Original Message-----
> > >>> From: Fariborz Navidan <md...@gmail.com>
> > >>> Sent: 28 August 2019 12:29
> > >>> To: users@cloudstack.apache.org
> > >>> Subject: Unable to start System VMs
> > >>>
> > >>> Hello,
> > >>>
> > >>> New ACS management is unable to start System VMs. Below are logs. ACS
> > is
> > >>> still poor in error handling and reporting. It should be able to
> retyrn
> > >>> clear error messages inside UI.
> > >>>
> > >>> 2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
> > >>> expired async-jobs
> > >>> 2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup
> > >>> expired
> > >>> async-jobs
> > >>> 2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
> > >>> expired async-jobs
> > >>> 2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup
> > >>> expired
> > >>> async-jobs
> > >>> 2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
> > >>> 137.74.35.102 -- GET
> > >>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
> > >>> 2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170)
> > ===END===
> > >>> 137.74.35.102 -- GET
> > >>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
> > >>> 2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START===
> > >>> 137.74.35.102
> > >>> -- GET
> > >>>
> > >>>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
> > >>> 2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5)
> ===END===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
> > >>> 2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
> > >>> 2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933)
> > ===END===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
> > >>> 2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START===
> > >>> 137.74.35.102
> > >>> -- GET
> > command=listApis&response=json&name=listVmwareDcs&_=1566988128967
> > >>> 2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api
> > >>> Discovery
> > >>> plugin was unable to find an api by that name or process any apis
> > >>> 2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719)
> ===END===
> > >>> 137.74.35.102 -- GET
> > >>> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
> > >>> 2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
> > >>> expired async-jobs
> > >>> 2019-08-28 13:22:14,852 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers
> > to
> > >>> update status.
> > >>> 2019-08-28 13:22:14,854 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC
> > >>> networks
> > >>> to update Redundant State.
> > >>> 2019-08-28 13:22:14,856 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0
> networks
> > to
> > >>> update RvR status.
> > >>> 2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup
> > >>> expired
> > >>> async-jobs
> > >>> 2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
> > >>> 2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8)
> > ===END===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
> > >>> 2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START===
> > >>> 137.74.35.102
> > >>> -- GET
> > >>>
> > >>>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
> > >>> 2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41)
> ===END===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
> > >>> 2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
> > >>> 2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05)
> > ===END===
> > >>> 137.74.35.102 -- GET
> > >>>
> > >>>
> >
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
> > >>> 2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START===
> > >>> 137.74.35.102
> > >>> -- GET
> > command=listApis&response=json&name=listVmwareDcs&_=1566988128971
> > >>> 2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api
> > >>> Discovery
> > >>> plugin was unable to find an api by that name or process any apis
> > >>> 2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5)
> ===END===
> > >>> 137.74.35.102 -- GET
> > >>> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
> > >>> 2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl]
> > >>> (ClusteredAgentManager
> > >>> Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for
> > >>> reconnect
> > >>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
> > >>> (ClusteredAgentManager
> > >>> Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts
> suitable
> > >>> for
> > >>> reconnect
> > >>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
> > >>> (ClusteredAgentManager
> > >>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters
> > already
> > >>> owned by this management server
> > >>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
> > >>> (ClusteredAgentManager
> > >>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
> > >>> clusters
> > >>> already owned by this management server
> > >>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
> > >>> (ClusteredAgentManager
> > >>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not
> > >>> owned
> > >>> by any management server
> > >>> 2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl]
> > >>> (ClusteredAgentManager
> > >>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
> > >>> clusters
> > >>> not owned by any management server
> > >>> 2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
> > >>> 137.74.35.102 -- GET
> > >>> command=listInfrastructure&response=json&_=1566988128972
> > >>> 2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef)
> > ===END===
> > >>> 137.74.35.102 -- GET
> > >>> command=listInfrastructure&response=json&_=1566988128972
> > >>> 2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
> > >>> expired async-jobs
> > >>> 2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup
> > >>> expired
> > >>> async-jobs
> > >>> 2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>> (AgentManager-Handler-3:null) (logid:) Ping from 1(
> fr-kvm1.r9host.com)
> > >>> 2019-08-28 13:22:24,957 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >>> (AgentManager-Handler-3:null) (logid:) Process host VM state report
> > from
> > >>> ping process. host: 1
> > >>> 2019-08-28 13:22:24,957 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >>> (AgentManager-Handler-3:null) (logid:) Process VM state report. host:
> > 1,
> > >>> number of records in report: 0
> > >>> 2019-08-28 13:22:24,957 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
> > >>> report. host: 1
> > >>> 2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
> > >>> (StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor
> is
> > >>> running...
> > >>> 2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START===
> > >>> 137.74.35.102
> > >>> -- GET command=listInfrastructure&response=json&_=1566988128973
> > >>> 2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
> > >>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs
> > from
> > >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> > >>> allowed
> > >>> to perform API calls: 0.0.0.0/0,::/0
> > >>> 2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
> > >>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0)
> ===END===
> > >>> 137.74.35.102 -- GET
> > >>> command=listInfrastructure&response=json&_=1566988128973
> > >>> 2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
> > >>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector
> is
> > >>> running...
> > >>> 2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
> > >>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq
> > >>> 1-5778962746846478713:
> > >>> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >>> 2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
> > >>> running...
> > >>> 2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no
> secondary
> > >>> storage VM for secondary storage host Secondary1
> > >>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20)
> > >>> getCommandHostDelegation:
> > >>> class com.cloud.agent.api.GetStorageStatsCommand
> > >>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
> > >>> default host to execute commands because the command is not of Copy
> > type.
> > >>> 2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
> > >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq
> > >>> 1-5778962746846478714:
> > >>> Received: { Ans: , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com
> > ),
> > >>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> > >>> 2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
> > >>> expired async-jobs
> > >>> 2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup
> > >>> expired
> > >>> async-jobs
> > >>> 2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to
> launch
> > >>> secondary storage VM
> > >>> 2019-08-28 13:22:40,562 INFO
> > >>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary
> > storage
> > >>> vms found in datacenter id=1, starting one
> > >>> 2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary
> > storage
> > >>> vm is available, need to allocate a new secondary storage vm
> > >>> 2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage
> > vm
> > >>> from a newly started instance for request from data center : 1
> > >>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> > >>> configuration for offering [Network Offering
> > >>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]:
> Ntwk[203|Guest|6]
> > >>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> > >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> > >>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> > >>> configuration for offering [Network Offering
> > >>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
> > >>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> > >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> > >>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> > >>> configuration for offering [Network Offering
> > >>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
> > >>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> > >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> > >>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > >>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to
> > launch
> >
>
Re: Unable to start System VMs
Posted by Andrija Panic <an...@gmail.com>.
I Don't see anything attached from agent.log? Make sure to put agent in
debug mode before posting logs (google for "cloudstack agent debug mode",
there is a simple sed command, restart agent afterwards).
Andrija
On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <md...@gmail.com> wrote:
> Hi,
>
> Also this is agent log
>
> On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Any idea?
> >
> > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> >> Thanks. I changed VLAN ID and now they are kept starting for along time.
> >> Latest logs:
> >>
> >>
> >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been pending
> for
> >> 959 seconds
> >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been pending
> for
> >> 357 seconds
> >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:34:14,852 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 routers to
> >> update status.
> >> 2019-08-28 14:34:14,854 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC
> networks
> >> to update Redundant State.
> >> 2019-08-28 14:34:14,857 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 networks
> to
> >> update RvR status.
> >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598229
> >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) ===END===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598229
> >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Resetting
> hosts
> >> suitable for reconnect
> >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
> >> resetting hosts suitable for reconnect
> >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring
> hosts
> >> for clusters already owned by this management server
> >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
> >> acquiring hosts for clusters already owned by this management server
> >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring
> hosts
> >> for clusters not owned by any management server
> >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
> >> acquiring hosts for clusters not owned by any management server
> >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
> >> 137.74.35.102 -- GET
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) ===END===
> >> 137.74.35.102 -- GET
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-15:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> >> 2019-08-28 14:34:24,974 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) Process host VM state report
> from
> >> ping process. host: 1
> >> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) Process VM state report. host:
> 1,
> >> number of records in report: 1
> >> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1, vm id:
> >> 103, power state: PowerOn
> >> 2019-08-28 14:34:24,976 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> host:
> >> 1, vm id: 103, power state: PowerOn
> >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 103, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) Run missing VM report. current
> >> time: 1566995664980
> >> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
> >> id: 101, power state: PowerReportMissing, last state update:
> 1566995544000
> >> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since last
> state
> >> update(120980ms) has passed graceful period
> >> 2019-08-28 14:34:24,982 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> host:
> >> 1, vm id: 101, power state: PowerReportMissing
> >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 101, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
> >> id: 102, power state: PowerReportMissing, last state update:
> 1566995544000
> >> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since last
> state
> >> update(120980ms) has passed graceful period
> >> 2019-08-28 14:34:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) VM state report is updated.
> host:
> >> 1, vm id: 102, power state: PowerReportMissing
> >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 102, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:34:24,987 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-15:null) (logid:) Done with process of VM state
> >> report. host: 1
> >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598231
> >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) ===END===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598231
> >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
> >> 137.74.35.102 -- GET
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) ===END===
> >> 137.74.35.102 -- GET
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling Monitor is
> >> running...
> >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
> >> Discovery plugin was unable to find an api by that name or process any
> apis
> >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) ===END===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector is
> >> running...
> >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq
> 1-5778962746846479041:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector is
> >> running...
> >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no secondary
> >> storage VM for secondary storage host Secondary1
> >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc)
> getCommandHostDelegation:
> >> class com.cloud.agent.api.GetStorageStatsCommand
> >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning the
> >> default host to execute commands because the command is not of Copy
> type.
> >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq
> 1-5778962746846479042:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
> >> Discovery plugin was unable to find an api by that name or process any
> apis
> >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) ===END===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598240
> >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) ===END===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598240
> >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:34:44,852 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 routers to
> >> update status.
> >> 2019-08-28 14:34:44,855 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC
> networks
> >> to update Redundant State.
> >> 2019-08-28 14:34:44,857 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 networks
> to
> >> update RvR status.
> >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:48,215 WARN [c.c.a.d.ParamGenericValidationWorker]
> >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) Received
> >> unknown parameters for command listSystemVms. Unknown parameters :
> listall
> >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
> >> 137.74.35.102 -- GET
> >> command=listHosts&response=json&details=min&_=1566995598242
> >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >>>Searching
> >> for hosts>>>
> >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >> >>>Generating Response>>>
> >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) ===END===
> >> 137.74.35.102 -- GET
> >> command=listHosts&response=json&details=min&_=1566995598242
> >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598245
> >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) ===END===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598245
> >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
> >> 137.74.35.102 -- GET
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) ===END===
> >> 137.74.35.102 -- GET
> >> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
> >> Discovery plugin was unable to find an api by that name or process any
> apis
> >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) ===END===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been pending
> for
> >> 1019 seconds
> >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been pending
> for
> >> 417 seconds
> >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
> >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started resource
> >> counters recalculation periodic task.
> >> 2019-08-28 14:35:14,848 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External
> devices
> >> are not used. Skipping external device usage collection
> >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot
> scheduler.poll is
> >> being called at 2019-08-28 12:35:14 GMT
> >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:35:14,857 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 routers to
> >> update status.
> >> 2019-08-28 14:35:14,857 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterMonitor-1:ctx-7cb475c8)
> >> (logid:cc3e421d) Found 0 running routers.
> >> 2019-08-28 14:35:14,858 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC
> networks
> >> to update Redundant State.
> >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to be
> >> executed at 2019-08-28 12:35:14 GMT
> >> 2019-08-28 14:35:14,858 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 networks
> to
> >> update RvR status.
> >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
> >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4)
> NetworkGarbageCollector
> >> uses '600' seconds for GC interval.
> >> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
> >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage server
> >> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
> >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true,
> heartbeat:
> >> Wed Aug 28 14:35:12 CEST 2019
> >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
> >> Discovery plugin was unable to find an api by that name or process any
> apis
> >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) ===END===
> >> 137.74.35.102 -- GET
> >> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598254
> >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) ===END===
> >> 137.74.35.102 -- GET
> >> command=listInfrastructure&response=json&_=1566995598254
> >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager
> is
> >> running and getting the updates from LB providers and updating service
> >> status
> >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager
> is
> >> running and getting the updates from LB providers and updating service
> >> status
> >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-1:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> >> 2019-08-28 14:35:25,005 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) Process host VM state report from
> >> ping process. host: 1
> >> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) Process VM state report. host: 1,
> >> number of records in report: 1
> >> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm id:
> >> 103, power state: PowerOn
> >> 2019-08-28 14:35:25,007 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) VM state report is updated.
> host: 1,
> >> vm id: 103, power state: PowerOn
> >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-1:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 103, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:35:25,011 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) Run missing VM report. current
> time:
> >> 1566995725011
> >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm
> id:
> >> 101, power state: PowerReportMissing, last state update: 1566995664000
> >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last
> state
> >> update(61011ms) has not passed graceful period yet
> >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm
> id:
> >> 102, power state: PowerReportMissing, last state update: 1566995664000
> >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last
> state
> >> update(61011ms) has not passed graceful period yet
> >> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-1:null) (logid:) Done with process of VM state
> >> report. host: 1
> >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:26,390 WARN [c.c.a.d.ParamGenericValidationWorker]
> >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) Received
> >> unknown parameters for command listSystemVms. Unknown parameters :
> listall
> >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
> >> 137.74.35.102 -- GET
> >> command=listHosts&response=json&details=min&_=1566995598256
> >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
> >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> >>>Searching
> >> for hosts>>>
> >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
> >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> >> >>>Generating Response>>>
> >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) ===END===
> >> 137.74.35.102 -- GET
> >> command=listHosts&response=json&details=min&_=1566995598256
> >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq
> 1-5778962746846479043:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
> >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from executing
> >> class com.cloud.agent.api.GetVolumeStatsCommand:
> >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling Monitor is
> >> running...
> >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector is
> >> running...
> >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq
> 1-5778962746846479044:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector is
> >> running...
> >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no secondary
> >> storage VM for secondary storage host Secondary1
> >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42)
> getCommandHostDelegation:
> >> class com.cloud.agent.api.GetStorageStatsCommand
> >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
> >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning the
> >> default host to execute commands because the command is not of Copy
> type.
> >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq
> 1-5778962746846479045:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> >> 2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Checking
> if
> >> any host reservation can be released ...
> >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:35:44,852 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 routers to
> >> update status.
> >> 2019-08-28 14:35:44,854 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC
> networks
> >> to update Redundant State.
> >> 2019-08-28 14:35:44,856 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 networks
> to
> >> update RvR status.
> >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Cannot
> >> release reservation, Found 3 VMs Running on host 1
> >> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done
> running
> >> HostReservationReleaseChecker ...
> >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity Checker
> >> ...
> >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating system
> >> capacity
> >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram
> capacity
> >> update
> >> 2019-08-28 14:35:44,896 DEBUG [c.c.c.CapacityManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 3 VMs on host 1
> >> 2019-08-28 14:35:44,904 DEBUG [c.c.c.CapacityManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 0 VM, not running
> on
> >> host 1
> >> 2019-08-28 14:35:44,907 DEBUG [c.c.c.CapacityManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
> >> capacity, host:1 usedCpuCore: 3 reservedCpuCore: 0
> >> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
> >> capacity, host:1 usedCpu: 1500 reservedCpu: 0
> >> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate
> memory
> >> capacity, host:1 usedMem: 1879048192 reservedMem: 0
> >> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing cpu/ram
> >> capacity update
> >> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing storage
> capacity
> >> update
> >> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found storage pool
> Primary1
> >> of type SharedMountPoint
> >> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Total over provisioned
> >> capacity of the pool Primary1 id: 1 is 984895062016
> >> 2019-08-28 14:35:44,920 DEBUG [c.c.s.StorageManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Successfully set
> Capacity -
> >> 984895062016 for capacity type - 3 , DataCenterId - 1, HostOrPoolId - 1,
> >> PodId 1
> >> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing storage
> >> capacity update
> >> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity
> updates
> >> for public ip and Vlans
> >> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done capacity updates
> for
> >> public ip and Vlans
> >> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity
> updates
> >> for private ip
> >> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing capacity
> >> updates for private ip
> >> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done recalculating
> system
> >> capacity
> >> 2019-08-28 14:35:44,982 DEBUG [c.c.a.AlertManagerImpl]
> >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done running Capacity
> >> Checker ...
> >> 2019-08-28 14:35:49,922 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Resetting
> hosts
> >> suitable for reconnect
> >> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
> >> resetting hosts suitable for reconnect
> >> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring
> hosts
> >> for clusters already owned by this management server
> >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
> >> acquiring hosts for clusters already owned by this management server
> >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring
> hosts
> >> for clusters not owned by any management server
> >> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
> >> acquiring hosts for clusters not owned by any management server
> >> 2019-08-28 14:35:49,953 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-5e476ef2) (logid:be235fb1) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
> >> 2019-08-28 14:35:49,961 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:50,014 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
> >> 2019-08-28 14:35:50,633 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-3f36683f) (logid:7cc15823) ===START===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
> >> 2019-08-28 14:35:50,643 DEBUG [c.c.a.ApiServer]
> >> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) CIDRs
> from
> >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> allowed
> >> to perform API calls: 0.0.0.0/0,::/0
> >> 2019-08-28 14:35:50,698 DEBUG [c.c.a.ApiServlet]
> >> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) ===END===
> >> 137.74.35.102 -- GET
> >>
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
> >> 2019-08-28 14:35:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:35:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:36:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:36:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-318) has been pending
> for
> >> 1079 seconds
> >> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> >> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-320) has been pending
> for
> >> 477 seconds
> >> 2019-08-28 14:36:14,855 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:36:14,855 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 routers to
> >> update status.
> >> 2019-08-28 14:36:14,857 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 VPC
> networks
> >> to update Redundant State.
> >> 2019-08-28 14:36:14,859 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 networks
> to
> >> update RvR status.
> >> 2019-08-28 14:36:14,864 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:36:24,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:36:24,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:36:24,972 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-7:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> >> 2019-08-28 14:36:24,972 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) Process host VM state report from
> >> ping process. host: 1
> >> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) Process VM state report. host: 1,
> >> number of records in report: 1
> >> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) VM state report. host: 1, vm id:
> >> 103, power state: PowerOn
> >> 2019-08-28 14:36:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
> host: 1,
> >> vm id: 103, power state: PowerOn
> >> 2019-08-28 14:36:24,976 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 103, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:36:24,978 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) Run missing VM report. current
> time:
> >> 1566995784978
> >> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm
> id:
> >> 101, power state: PowerReportMissing, last state update: 1566995664000
> >> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) vm id: 101 - time since last
> state
> >> update(120978ms) has passed graceful period
> >> 2019-08-28 14:36:24,981 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
> host: 1,
> >> vm id: 101, power state: PowerReportMissing
> >> 2019-08-28 14:36:24,981 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 101, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm
> id:
> >> 102, power state: PowerReportMissing, last state update: 1566995664000
> >> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) vm id: 102 - time since last
> state
> >> update(120978ms) has passed graceful period
> >> 2019-08-28 14:36:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) VM state report is updated.
> host: 1,
> >> vm id: 102, power state: PowerReportMissing
> >> 2019-08-28 14:36:24,984 INFO [c.c.v.VirtualMachineManagerImpl]
> >> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
> >> working on the VM. vm id: 102, postpone power-change report by resetting
> >> power-change counters
> >> 2019-08-28 14:36:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-7:null) (logid:) Done with process of VM state
> >> report. host: 1
> >> 2019-08-28 14:36:30,181 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-6:ctx-9c8e3bed) (logid:9058cad8) AutoScaling Monitor is
> >> running...
> >> 2019-08-28 14:36:34,848 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) Begin cleanup
> >> expired async-jobs
> >> 2019-08-28 14:36:34,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) End cleanup
> expired
> >> async-jobs
> >> 2019-08-28 14:36:37,458 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) HostStatsCollector is
> >> running...
> >> 2019-08-28 14:36:37,515 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) Seq
> 1-5778962746846479046:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> 2019-08-28 14:36:38,661 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) StorageCollector is
> >> running...
> >> 2019-08-28 14:36:38,671 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) There is no secondary
> >> storage VM for secondary storage host Secondary1
> >> 2019-08-28 14:36:38,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca)
> getCommandHostDelegation:
> >> class com.cloud.agent.api.GetStorageStatsCommand
> >> 2019-08-28 14:36:38,675 DEBUG [c.c.h.XenServerGuru]
> >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) We are returning the
> >> default host to execute commands because the command is not of Copy
> type.
> >> 2019-08-28 14:36:38,729 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) Seq
> 1-5778962746846479047:
> >> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >>
> >> On Wed, Aug 28, 2019 at 4:04 PM Paul Angus <pa...@shapeblue.com>
> >> wrote:
> >>
> >>> The error is here Fariborz,
> >>>
> >>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
> >>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> >>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed
> to
> >>> create vnet 100: device eth1.100 is already a member of a bridge; can't
> >>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
> >>> breth1-100
> >>>
> >>> you're already using a VLAN that you've also asked CloudStack to use.
> >>>
> >>>
> >>>
> >>> paul.angus@shapeblue.com
> >>> www.shapeblue.com
> >>> Amadeus House, Floral Street, London WC2E 9DPUK
> >>> @shapeblue
> >>>
> >>>
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Fariborz Navidan <md...@gmail.com>
> >>> Sent: 28 August 2019 12:29
> >>> To: users@cloudstack.apache.org
> >>> Subject: Unable to start System VMs
> >>>
> >>> Hello,
> >>>
> >>> New ACS management is unable to start System VMs. Below are logs. ACS
> is
> >>> still poor in error handling and reporting. It should be able to retyrn
> >>> clear error messages inside UI.
> >>>
> >>> 2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
> >>> expired async-jobs
> >>> 2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup
> >>> expired
> >>> async-jobs
> >>> 2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
> >>> expired async-jobs
> >>> 2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup
> >>> expired
> >>> async-jobs
> >>> 2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
> >>> 137.74.35.102 -- GET
> >>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
> >>> 2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170)
> ===END===
> >>> 137.74.35.102 -- GET
> >>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
> >>> 2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START===
> >>> 137.74.35.102
> >>> -- GET
> >>>
> >>>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
> >>> 2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) ===END===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
> >>> 2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
> >>> 2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933)
> ===END===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
> >>> 2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START===
> >>> 137.74.35.102
> >>> -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
> >>> 2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api
> >>> Discovery
> >>> plugin was unable to find an api by that name or process any apis
> >>> 2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) ===END===
> >>> 137.74.35.102 -- GET
> >>> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
> >>> 2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
> >>> expired async-jobs
> >>> 2019-08-28 13:22:14,852 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers
> to
> >>> update status.
> >>> 2019-08-28 13:22:14,854 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC
> >>> networks
> >>> to update Redundant State.
> >>> 2019-08-28 13:22:14,856 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 networks
> to
> >>> update RvR status.
> >>> 2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup
> >>> expired
> >>> async-jobs
> >>> 2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
> >>> 2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8)
> ===END===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
> >>> 2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START===
> >>> 137.74.35.102
> >>> -- GET
> >>>
> >>>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
> >>> 2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) ===END===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
> >>> 2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
> >>> 2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05)
> ===END===
> >>> 137.74.35.102 -- GET
> >>>
> >>>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
> >>> 2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START===
> >>> 137.74.35.102
> >>> -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
> >>> 2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api
> >>> Discovery
> >>> plugin was unable to find an api by that name or process any apis
> >>> 2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) ===END===
> >>> 137.74.35.102 -- GET
> >>> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
> >>> 2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl]
> >>> (ClusteredAgentManager
> >>> Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for
> >>> reconnect
> >>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
> >>> (ClusteredAgentManager
> >>> Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts suitable
> >>> for
> >>> reconnect
> >>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
> >>> (ClusteredAgentManager
> >>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters
> already
> >>> owned by this management server
> >>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
> >>> (ClusteredAgentManager
> >>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
> >>> clusters
> >>> already owned by this management server
> >>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
> >>> (ClusteredAgentManager
> >>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not
> >>> owned
> >>> by any management server
> >>> 2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl]
> >>> (ClusteredAgentManager
> >>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
> >>> clusters
> >>> not owned by any management server
> >>> 2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
> >>> 137.74.35.102 -- GET
> >>> command=listInfrastructure&response=json&_=1566988128972
> >>> 2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef)
> ===END===
> >>> 137.74.35.102 -- GET
> >>> command=listInfrastructure&response=json&_=1566988128972
> >>> 2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
> >>> expired async-jobs
> >>> 2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup
> >>> expired
> >>> async-jobs
> >>> 2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
> >>> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> >>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >>> (AgentManager-Handler-3:null) (logid:) Process host VM state report
> from
> >>> ping process. host: 1
> >>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >>> (AgentManager-Handler-3:null) (logid:) Process VM state report. host:
> 1,
> >>> number of records in report: 0
> >>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> >>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
> >>> report. host: 1
> >>> 2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
> >>> (StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor is
> >>> running...
> >>> 2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START===
> >>> 137.74.35.102
> >>> -- GET command=listInfrastructure&response=json&_=1566988128973
> >>> 2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
> >>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs
> from
> >>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
> >>> allowed
> >>> to perform API calls: 0.0.0.0/0,::/0
> >>> 2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
> >>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) ===END===
> >>> 137.74.35.102 -- GET
> >>> command=listInfrastructure&response=json&_=1566988128973
> >>> 2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
> >>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector is
> >>> running...
> >>> 2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
> >>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq
> >>> 1-5778962746846478713:
> >>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >>> 2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
> >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
> >>> running...
> >>> 2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
> >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no secondary
> >>> storage VM for secondary storage host Secondary1
> >>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20)
> >>> getCommandHostDelegation:
> >>> class com.cloud.agent.api.GetStorageStatsCommand
> >>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
> >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
> >>> default host to execute commands because the command is not of Copy
> type.
> >>> 2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
> >>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq
> >>> 1-5778962746846478714:
> >>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com
> ),
> >>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> >>> 2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
> >>> expired async-jobs
> >>> 2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup
> >>> expired
> >>> async-jobs
> >>> 2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to launch
> >>> secondary storage VM
> >>> 2019-08-28 13:22:40,562 INFO
> >>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary
> storage
> >>> vms found in datacenter id=1, starting one
> >>> 2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary
> storage
> >>> vm is available, need to allocate a new secondary storage vm
> >>> 2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage
> vm
> >>> from a newly started instance for request from data center : 1
> >>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> >>> configuration for offering [Network Offering
> >>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
> >>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> >>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> >>> configuration for offering [Network Offering
> >>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
> >>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> >>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> >>> configuration for offering [Network Offering
> >>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
> >>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> >>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> >>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to
> launch
>
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
Hi,
Also this is agent log
On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan <md...@gmail.com>
wrote:
> Any idea?
>
> On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
>> Thanks. I changed VLAN ID and now they are kept starting for along time.
>> Latest logs:
>>
>>
>> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup expired
>> async-jobs
>> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been pending for
>> 959 seconds
>> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been pending for
>> 357 seconds
>> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:34:14,852 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 routers to
>> update status.
>> 2019-08-28 14:34:14,854 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 14:34:14,857 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 networks to
>> update RvR status.
>> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup expired
>> async-jobs
>> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598229
>> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598229
>> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Resetting hosts
>> suitable for reconnect
>> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
>> resetting hosts suitable for reconnect
>> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring hosts
>> for clusters already owned by this management server
>> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
>> acquiring hosts for clusters already owned by this management server
>> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring hosts
>> for clusters not owned by any management server
>> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed
>> acquiring hosts for clusters not owned by any management server
>> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
>> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
>> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup expired
>> async-jobs
>> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>> 2019-08-28 14:34:24,974 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) Process host VM state report from
>> ping process. host: 1
>> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) Process VM state report. host: 1,
>> number of records in report: 1
>> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1, vm id:
>> 103, power state: PowerOn
>> 2019-08-28 14:34:24,976 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
>> 1, vm id: 103, power state: PowerOn
>> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 103, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) Run missing VM report. current
>> time: 1566995664980
>> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
>> id: 101, power state: PowerReportMissing, last state update: 1566995544000
>> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since last state
>> update(120980ms) has passed graceful period
>> 2019-08-28 14:34:24,982 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
>> 1, vm id: 101, power state: PowerReportMissing
>> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 101, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
>> id: 102, power state: PowerReportMissing, last state update: 1566995544000
>> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since last state
>> update(120980ms) has passed graceful period
>> 2019-08-28 14:34:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
>> 1, vm id: 102, power state: PowerReportMissing
>> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 102, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:34:24,987 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-15:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598231
>> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598231
>> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
>> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
>> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling Monitor is
>> running...
>> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
>> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
>> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
>> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) ===END===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
>> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
>> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
>> Discovery plugin was unable to find an api by that name or process any apis
>> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) ===END===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
>> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup expired
>> async-jobs
>> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
>> 137.74.35.102 -- GET
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
>> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) ===END===
>> 137.74.35.102 -- GET
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
>> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector is
>> running...
>> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
>> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq 1-5778962746846479041:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector is
>> running...
>> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no secondary
>> storage VM for secondary storage host Secondary1
>> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) getCommandHostDelegation:
>> class com.cloud.agent.api.GetStorageStatsCommand
>> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
>> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning the
>> default host to execute commands because the command is not of Copy type.
>> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
>> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq 1-5778962746846479042:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
>> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
>> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
>> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) ===END===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
>> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
>> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
>> Discovery plugin was unable to find an api by that name or process any apis
>> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) ===END===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
>> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598240
>> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598240
>> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:34:44,852 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 routers to
>> update status.
>> 2019-08-28 14:34:44,855 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 14:34:44,857 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 networks to
>> update RvR status.
>> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup expired
>> async-jobs
>> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
>> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:48,215 WARN [c.c.a.d.ParamGenericValidationWorker]
>> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) Received
>> unknown parameters for command listSystemVms. Unknown parameters : listall
>> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
>> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
>> 137.74.35.102 -- GET
>> command=listHosts&response=json&details=min&_=1566995598242
>> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
>> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) >>>Searching
>> for hosts>>>
>> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
>> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>> >>>Generating Response>>>
>> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) ===END===
>> 137.74.35.102 -- GET
>> command=listHosts&response=json&details=min&_=1566995598242
>> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup expired
>> async-jobs
>> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
>> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
>> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
>> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
>> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598245
>> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598245
>> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup expired
>> async-jobs
>> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
>> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
>> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
>> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
>> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
>> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) ===END===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
>> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
>> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
>> Discovery plugin was unable to find an api by that name or process any apis
>> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) ===END===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
>> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been pending for
>> 1019 seconds
>> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been pending for
>> 417 seconds
>> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
>> 137.74.35.102 -- GET
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
>> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) ===END===
>> 137.74.35.102 -- GET
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
>> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
>> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started resource
>> counters recalculation periodic task.
>> 2019-08-28 14:35:14,848 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
>> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External devices
>> are not used. Skipping external device usage collection
>> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot scheduler.poll is
>> being called at 2019-08-28 12:35:14 GMT
>> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup expired
>> async-jobs
>> 2019-08-28 14:35:14,857 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 routers to
>> update status.
>> 2019-08-28 14:35:14,857 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-7cb475c8)
>> (logid:cc3e421d) Found 0 running routers.
>> 2019-08-28 14:35:14,858 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to be
>> executed at 2019-08-28 12:35:14 GMT
>> 2019-08-28 14:35:14,858 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 networks to
>> update RvR status.
>> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
>> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4) NetworkGarbageCollector
>> uses '600' seconds for GC interval.
>> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
>> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage server
>> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
>> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true, heartbeat:
>> Wed Aug 28 14:35:12 CEST 2019
>> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
>> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
>> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
>> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) ===END===
>> 137.74.35.102 -- GET
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
>> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
>> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
>> Discovery plugin was unable to find an api by that name or process any apis
>> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) ===END===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
>> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598254
>> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566995598254
>> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup expired
>> async-jobs
>> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager is
>> running and getting the updates from LB providers and updating service
>> status
>> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager is
>> running and getting the updates from LB providers and updating service
>> status
>> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>> 2019-08-28 14:35:25,005 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) Process host VM state report from
>> ping process. host: 1
>> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) Process VM state report. host: 1,
>> number of records in report: 1
>> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm id:
>> 103, power state: PowerOn
>> 2019-08-28 14:35:25,007 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) VM state report is updated. host: 1,
>> vm id: 103, power state: PowerOn
>> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 103, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:35:25,011 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) Run missing VM report. current time:
>> 1566995725011
>> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm id:
>> 101, power state: PowerReportMissing, last state update: 1566995664000
>> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last state
>> update(61011ms) has not passed graceful period yet
>> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm id:
>> 102, power state: PowerReportMissing, last state update: 1566995664000
>> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last state
>> update(61011ms) has not passed graceful period yet
>> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-1:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
>> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:26,390 WARN [c.c.a.d.ParamGenericValidationWorker]
>> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) Received
>> unknown parameters for command listSystemVms. Unknown parameters : listall
>> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
>> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
>> 137.74.35.102 -- GET
>> command=listHosts&response=json&details=min&_=1566995598256
>> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
>> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) >>>Searching
>> for hosts>>>
>> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
>> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>> >>>Generating Response>>>
>> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) ===END===
>> 137.74.35.102 -- GET
>> command=listHosts&response=json&details=min&_=1566995598256
>> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
>> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq 1-5778962746846479043:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
>> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
>> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from executing
>> class com.cloud.agent.api.GetVolumeStatsCommand:
>> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling Monitor is
>> running...
>> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
>> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
>> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
>> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
>> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup expired
>> async-jobs
>> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector is
>> running...
>> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
>> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq 1-5778962746846479044:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector is
>> running...
>> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no secondary
>> storage VM for secondary storage host Secondary1
>> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) getCommandHostDelegation:
>> class com.cloud.agent.api.GetStorageStatsCommand
>> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
>> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning the
>> default host to execute commands because the command is not of Copy type.
>> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
>> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq 1-5778962746846479045:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
>> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
>> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
>> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
>> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
>> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
>> 2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Checking if
>> any host reservation can be released ...
>> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:35:44,852 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 routers to
>> update status.
>> 2019-08-28 14:35:44,854 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 14:35:44,856 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 networks to
>> update RvR status.
>> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Cannot
>> release reservation, Found 3 VMs Running on host 1
>> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done running
>> HostReservationReleaseChecker ...
>> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup expired
>> async-jobs
>> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity Checker
>> ...
>> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating system
>> capacity
>> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram capacity
>> update
>> 2019-08-28 14:35:44,896 DEBUG [c.c.c.CapacityManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 3 VMs on host 1
>> 2019-08-28 14:35:44,904 DEBUG [c.c.c.CapacityManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 0 VM, not running on
>> host 1
>> 2019-08-28 14:35:44,907 DEBUG [c.c.c.CapacityManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
>> capacity, host:1 usedCpuCore: 3 reservedCpuCore: 0
>> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
>> capacity, host:1 usedCpu: 1500 reservedCpu: 0
>> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate memory
>> capacity, host:1 usedMem: 1879048192 reservedMem: 0
>> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing cpu/ram
>> capacity update
>> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing storage capacity
>> update
>> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found storage pool Primary1
>> of type SharedMountPoint
>> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Total over provisioned
>> capacity of the pool Primary1 id: 1 is 984895062016
>> 2019-08-28 14:35:44,920 DEBUG [c.c.s.StorageManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Successfully set Capacity -
>> 984895062016 for capacity type - 3 , DataCenterId - 1, HostOrPoolId - 1,
>> PodId 1
>> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing storage
>> capacity update
>> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity updates
>> for public ip and Vlans
>> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done capacity updates for
>> public ip and Vlans
>> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity updates
>> for private ip
>> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing capacity
>> updates for private ip
>> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done recalculating system
>> capacity
>> 2019-08-28 14:35:44,982 DEBUG [c.c.a.AlertManagerImpl]
>> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done running Capacity
>> Checker ...
>> 2019-08-28 14:35:49,922 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Resetting hosts
>> suitable for reconnect
>> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
>> resetting hosts suitable for reconnect
>> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring hosts
>> for clusters already owned by this management server
>> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
>> acquiring hosts for clusters already owned by this management server
>> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Acquiring hosts
>> for clusters not owned by any management server
>> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager Timer:ctx-7f369e38) (logid:88e96986) Completed
>> acquiring hosts for clusters not owned by any management server
>> 2019-08-28 14:35:49,953 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-5e476ef2) (logid:be235fb1) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
>> 2019-08-28 14:35:49,961 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:50,014 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
>> 2019-08-28 14:35:50,633 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-3f36683f) (logid:7cc15823) ===START===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
>> 2019-08-28 14:35:50,643 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 14:35:50,698 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) ===END===
>> 137.74.35.102 -- GET
>> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
>> 2019-08-28 14:35:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:35:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) End cleanup expired
>> async-jobs
>> 2019-08-28 14:36:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:36:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) End cleanup expired
>> async-jobs
>> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-318) has been pending for
>> 1079 seconds
>> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
>> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-320) has been pending for
>> 477 seconds
>> 2019-08-28 14:36:14,855 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:36:14,855 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 routers to
>> update status.
>> 2019-08-28 14:36:14,857 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 14:36:14,859 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 networks to
>> update RvR status.
>> 2019-08-28 14:36:14,864 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) End cleanup expired
>> async-jobs
>> 2019-08-28 14:36:24,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:36:24,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) End cleanup expired
>> async-jobs
>> 2019-08-28 14:36:24,972 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>> 2019-08-28 14:36:24,972 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) Process host VM state report from
>> ping process. host: 1
>> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) Process VM state report. host: 1,
>> number of records in report: 1
>> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) VM state report. host: 1, vm id:
>> 103, power state: PowerOn
>> 2019-08-28 14:36:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
>> vm id: 103, power state: PowerOn
>> 2019-08-28 14:36:24,976 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 103, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:36:24,978 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) Run missing VM report. current time:
>> 1566995784978
>> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm id:
>> 101, power state: PowerReportMissing, last state update: 1566995664000
>> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) vm id: 101 - time since last state
>> update(120978ms) has passed graceful period
>> 2019-08-28 14:36:24,981 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
>> vm id: 101, power state: PowerReportMissing
>> 2019-08-28 14:36:24,981 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 101, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm id:
>> 102, power state: PowerReportMissing, last state update: 1566995664000
>> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) vm id: 102 - time since last state
>> update(120978ms) has passed graceful period
>> 2019-08-28 14:36:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
>> vm id: 102, power state: PowerReportMissing
>> 2019-08-28 14:36:24,984 INFO [c.c.v.VirtualMachineManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
>> working on the VM. vm id: 102, postpone power-change report by resetting
>> power-change counters
>> 2019-08-28 14:36:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-7:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2019-08-28 14:36:30,181 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-6:ctx-9c8e3bed) (logid:9058cad8) AutoScaling Monitor is
>> running...
>> 2019-08-28 14:36:34,848 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) Begin cleanup
>> expired async-jobs
>> 2019-08-28 14:36:34,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) End cleanup expired
>> async-jobs
>> 2019-08-28 14:36:37,458 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) HostStatsCollector is
>> running...
>> 2019-08-28 14:36:37,515 DEBUG [c.c.a.t.Request]
>> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) Seq 1-5778962746846479046:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2019-08-28 14:36:38,661 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) StorageCollector is
>> running...
>> 2019-08-28 14:36:38,671 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) There is no secondary
>> storage VM for secondary storage host Secondary1
>> 2019-08-28 14:36:38,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) getCommandHostDelegation:
>> class com.cloud.agent.api.GetStorageStatsCommand
>> 2019-08-28 14:36:38,675 DEBUG [c.c.h.XenServerGuru]
>> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) We are returning the
>> default host to execute commands because the command is not of Copy type.
>> 2019-08-28 14:36:38,729 DEBUG [c.c.a.t.Request]
>> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) Seq 1-5778962746846479047:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>>
>> On Wed, Aug 28, 2019 at 4:04 PM Paul Angus <pa...@shapeblue.com>
>> wrote:
>>
>>> The error is here Fariborz,
>>>
>>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
>>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>> breth1-100
>>>
>>> you're already using a VLAN that you've also asked CloudStack to use.
>>>
>>>
>>>
>>> paul.angus@shapeblue.com
>>> www.shapeblue.com
>>> Amadeus House, Floral Street, London WC2E 9DPUK
>>> @shapeblue
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Fariborz Navidan <md...@gmail.com>
>>> Sent: 28 August 2019 12:29
>>> To: users@cloudstack.apache.org
>>> Subject: Unable to start System VMs
>>>
>>> Hello,
>>>
>>> New ACS management is unable to start System VMs. Below are logs. ACS is
>>> still poor in error handling and reporting. It should be able to retyrn
>>> clear error messages inside UI.
>>>
>>> 2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
>>> 137.74.35.102 -- GET
>>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
>>> 2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) ===END===
>>> 137.74.35.102 -- GET
>>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
>>> 2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START===
>>> 137.74.35.102
>>> -- GET
>>>
>>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
>>> 2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) ===END===
>>> 137.74.35.102 -- GET
>>>
>>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
>>> 2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
>>> 137.74.35.102 -- GET
>>>
>>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
>>> 2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) ===END===
>>> 137.74.35.102 -- GET
>>>
>>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
>>> 2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START===
>>> 137.74.35.102
>>> -- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128967
>>> 2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api
>>> Discovery
>>> plugin was unable to find an api by that name or process any apis
>>> 2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) ===END===
>>> 137.74.35.102 -- GET
>>> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
>>> 2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:22:14,852 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers to
>>> update status.
>>> 2019-08-28 13:22:14,854 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC
>>> networks
>>> to update Redundant State.
>>> 2019-08-28 13:22:14,856 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 networks to
>>> update RvR status.
>>> 2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
>>> 137.74.35.102 -- GET
>>>
>>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
>>> 2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) ===END===
>>> 137.74.35.102 -- GET
>>>
>>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
>>> 2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START===
>>> 137.74.35.102
>>> -- GET
>>>
>>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
>>> 2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) ===END===
>>> 137.74.35.102 -- GET
>>>
>>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
>>> 2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
>>> 137.74.35.102 -- GET
>>>
>>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
>>> 2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) ===END===
>>> 137.74.35.102 -- GET
>>>
>>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
>>> 2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START===
>>> 137.74.35.102
>>> -- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128971
>>> 2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api
>>> Discovery
>>> plugin was unable to find an api by that name or process any apis
>>> 2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) ===END===
>>> 137.74.35.102 -- GET
>>> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
>>> 2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl]
>>> (ClusteredAgentManager
>>> Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for
>>> reconnect
>>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
>>> (ClusteredAgentManager
>>> Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts suitable
>>> for
>>> reconnect
>>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl]
>>> (ClusteredAgentManager
>>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters already
>>> owned by this management server
>>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
>>> (ClusteredAgentManager
>>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
>>> clusters
>>> already owned by this management server
>>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl]
>>> (ClusteredAgentManager
>>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not
>>> owned
>>> by any management server
>>> 2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl]
>>> (ClusteredAgentManager
>>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
>>> clusters
>>> not owned by any management server
>>> 2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128972
>>> 2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128972
>>> 2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
>>> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>>> (AgentManager-Handler-3:null) (logid:) Process host VM state report from
>>> ping process. host: 1
>>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>>> (AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
>>> number of records in report: 0
>>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
>>> report. host: 1
>>> 2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
>>> (StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor is
>>> running...
>>> 2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START===
>>> 137.74.35.102
>>> -- GET command=listInfrastructure&response=json&_=1566988128973
>>> 2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128973
>>> 2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
>>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector is
>>> running...
>>> 2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
>>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq
>>> 1-5778962746846478713:
>>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>>> 2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
>>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
>>> running...
>>> 2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
>>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no secondary
>>> storage VM for secondary storage host Secondary1
>>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>>> (StatsCollector-3:ctx-07390103) (logid:8f414a20)
>>> getCommandHostDelegation:
>>> class com.cloud.agent.api.GetStorageStatsCommand
>>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
>>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
>>> default host to execute commands because the command is not of Copy type.
>>> 2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
>>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq
>>> 1-5778962746846478714:
>>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>>> 2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to launch
>>> secondary storage VM
>>> 2019-08-28 13:22:40,562 INFO
>>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary storage
>>> vms found in datacenter id=1, starting one
>>> 2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary storage
>>> vm is available, need to allocate a new secondary storage vm
>>> 2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage vm
>>> from a newly started instance for request from data center : 1
>>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>>> configuration for offering [Network Offering
>>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
>>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>>> configuration for offering [Network Offering
>>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
>>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>>> configuration for offering [Network Offering
>>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
>>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to launch
>>> console proxy
>>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Expand console proxy
>>> standby
>>> capacity for zone France
>>> 2019-08-28 13:22:40,615 INFO [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) No stopped console proxy
>>> is
>>> available, need to allocate a new console proxy
>>> 2019-08-28 13:22:40,616 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Assign console proxy from
>>> a
>>> newly started instance for request from data center : 1
>>> 2019-08-28 13:22:40,623 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating entries for VM:
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:22:40,625 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nics for
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:22:40,628 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
>>> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[203|Guest|6] with
>>> requested
>>> profile NicProfile[0-0-null-null-null
>>> 2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
>>> configuration for offering [Network Offering
>>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
>>> 2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
>>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>>> 2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
>>> configuration for offering [Network Offering
>>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
>>> 2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
>>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>>> 2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
>>> configuration for offering [Network Offering
>>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
>>> 2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
>>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>>> 2019-08-28 13:22:40,660 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
>>> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[201|Control|3] with
>>> requested profile null
>>> 2019-08-28 13:22:40,668 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
>>> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[200|Management|2] with
>>> requested profile null
>>> 2019-08-28 13:22:40,673 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating entries for VM:
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:22:40,675 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nics for
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:22:40,676 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
>>> VM[ConsoleProxy|v-68-VM] in network Ntwk[203|Guest|6] with requested
>>> profile NicProfile[0-0-null-null-null
>>> 2019-08-28 13:22:40,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating disks for
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:22:40,681 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocation completed for VM:
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage
>>> vm
>>> alert
>>> 2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) New secondary storage vm
>>> created, zone: France, secStorageVm: s-67-VM, public IP: null, private
>>> IP:
>>> null
>>> 2019-08-28 13:22:40,688 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
>>> VM[ConsoleProxy|v-68-VM] in network Ntwk[201|Control|3] with requested
>>> profile null
>>> 2019-08-28 13:22:40,690 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Sync job-218 execution on
>>> object VmWorkJobQueue.67
>>> 2019-08-28 13:22:40,691 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
>>> VM[ConsoleProxy|v-68-VM] in network Ntwk[200|Management|2] with requested
>>> profile null
>>> 2019-08-28 13:22:40,694 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating disks for
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:22:40,696 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocation completed for
>>> VM:
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy
>>> alert
>>> 2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) New console proxy created,
>>> zone: France, proxy: v-68-VM, public IP: null, private IP: null
>>> 2019-08-28 13:22:40,701 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Sync job-219 execution on
>>> object VmWorkJobQueue.68
>>> 2019-08-28 13:22:40,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute
>>> sync-queue
>>> item: SyncQueueItemVO {id:194, queueId: 191, contentType: AsyncJob,
>>> contentId: 218, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
>>> lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
>>> 13:22:40 CEST 2019}
>>> 2019-08-28 13:22:40,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
>>> job-218
>>> 2019-08-28 13:22:40,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute
>>> sync-queue
>>> item: SyncQueueItemVO {id:195, queueId: 192, contentType: AsyncJob,
>>> contentId: 219, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
>>> lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
>>> 13:22:40 CEST 2019}
>>> 2019-08-28 13:22:40,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
>>> job-219
>>> 2019-08-28 13:22:40,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:d5030cfc) Add
>>> job-218 into job monitoring
>>> 2019-08-28 13:22:40,872 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:8371f7e8) Add
>>> job-219 into job monitoring
>>> 2019-08-28 13:22:40,877 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>>> Executing AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType:
>>> null,
>>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>>> null}
>>> 2019-08-28 13:22:40,877 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Run
>>> VM
>>> work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
>>> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Execute VM work job:
>>>
>>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:22:40,878 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>>> Executing AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType:
>>> null,
>>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>>> null}
>>> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Run
>>> VM
>>> work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
>>> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Execute VM work job:
>>>
>>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:22:40,880 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) VM state transitted from :Stopped to Starting with
>>> event:
>>> StartRequestedvm's original host id: null new host id: null host id
>>> before
>>> state transition: null
>>> 2019-08-28 13:22:40,880 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Successfully transitioned to start state for
>>> VM[SecondaryStorageVm|s-67-VM] reservation id =
>>> c608b88c-7d06-411e-bb84-f2d5e2b3390d
>>> 2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:22:40,881 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) VM state transitted from :Stopped to Starting with
>>> event:
>>> StartRequestedvm's original host id: null new host id: null host id
>>> before
>>> state transition: null
>>> 2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Successfully transitioned to start state for
>>> VM[ConsoleProxy|v-68-VM] reservation id =
>>> 6c836a9c-0d16-42c3-86c7-446febb1a189
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>>> pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Searching all possible resources under this Zone: 1
>>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Listing clusters in order of aggregate capacity, that
>>> have
>>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>>> pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
>>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
>>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Searching all possible resources under this Zone: 1
>>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that
>>> have
>>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>>> 2019-08-28 13:22:40,886 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:22:40,886 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:22:40,887 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>>> Ram=512
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>>> Ram=1024
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
>>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
>>> capacity for requested CPU: 500 and requested RAM: 536870912 ,
>>> cpuOverprovisioningFactor: 4.0
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU:
>>> 54000
>>> and CPU after applying overprovisioning: 216000
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
>>> CPU: 500
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
>>> Requested RAM: 536870912
>>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
>>> available
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from
>>> host:
>>> 1, used: 0, reserved: 0, actual total: 54000, total with
>>> overprovisioning:
>>> 216000; requested cpu:500,alloc_from_last_host?:false
>>> ,considerReservedCapacity?: true
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from
>>> host:
>>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>>> 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding
>>> to
>>> list: 1
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
>>> suitable hosts
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) We need to allocate new storagepool for this volume
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Calling StoragePoolAllocators to find suitable pools
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
>>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
>>> capacity for requested CPU: 500 and requested RAM: 1073741824 ,
>>> cpuOverprovisioningFactor: 4.0
>>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) System VMs will use shared storage for zone id=1
>>> 2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) LocalStoragePoolAllocator trying to find storage pool to
>>> fit the vm
>>> 2019-08-28 13:22:40,890 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) ClusterScopeStoragePoolAllocator looking for storage
>>> pool
>>> 2019-08-28 13:22:40,891 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
>>> pools will be ignored.
>>> 2019-08-28 13:22:40,891 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Found pools matching tags: [Pool[1|SharedMountPoint]]
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU:
>>> 54000
>>> and CPU after applying overprovisioning: 216000
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
>>> CPU: 500
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
>>> Requested RAM: 1073741824
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
>>> available
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from
>>> host:
>>> 1, used: 0, reserved: 0, actual total: 54000, total with
>>> overprovisioning:
>>> 216000; requested cpu:500,alloc_from_last_host?:false
>>> ,considerReservedCapacity?: true
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from
>>> host:
>>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>>> 1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding
>>> to
>>> list: 1
>>> 2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
>>> suitable hosts
>>> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking if storage pool is suitable, name: null
>>> ,poolId: 1
>>> 2019-08-28 13:22:40,891 INFO [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Storage pool null (1) does not supply IOPS capacity,
>>> assuming enough capacity
>>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
>>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) We need to allocate new storagepool for this volume
>>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Calling StoragePoolAllocators to find suitable pools
>>> 2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking pool 1 for storage, totalSize: 984895062016,
>>> usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold:
>>> 0.85
>>> 2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Destination pool id: 1
>>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) System VMs will use shared storage for zone id=1
>>> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) LocalStoragePoolAllocator trying to find storage pool to
>>> fit the vm
>>> 2019-08-28 13:22:40,892 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) ClusterScopeStoragePoolAllocator looking for storage
>>> pool
>>> 2019-08-28 13:22:40,892 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
>>> pools will be ignored.
>>> 2019-08-28 13:22:40,892 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Found pools matching tags: [Pool[1|SharedMountPoint]]
>>> 2019-08-28 13:22:40,893 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking if storage pool is suitable, name: null
>>> ,poolId: 1
>>> 2019-08-28 13:22:40,893 INFO [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Storage pool null (1) does not supply IOPS capacity,
>>> assuming enough capacity
>>> 2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking pool 1 for storage, totalSize: 984895062016,
>>> usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold:
>>> 0.85
>>> 2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Destination pool id: 1
>>> 2019-08-28 13:22:40,896 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Pool ID for the volume with ID 62 is null
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Found storage pool Primary1 of type SharedMountPoint
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Total capacity of the pool Primary1 with ID 1 is
>>> 984895062016
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking pool: 1 for storage allocation , maxSize :
>>> 984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
>>> disable threshold: 0.85
>>> 2019-08-28 13:22:40,897 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) ClusterScopeStoragePoolAllocator returning 1 suitable
>>> storage pools
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Trying to find a potenial host and associated storage
>>> pools from the suitable host/pool lists for this VM
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking if host: 1 can access any suitable storage pool
>>> for volume: ROOT
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Host: 1 can access pool: 1
>>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Pool ID for the volume with ID 63 is null
>>> 2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Found storage pool Primary1 of type SharedMountPoint
>>> 2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Total capacity of the pool Primary1 with ID 1 is
>>> 984895062016
>>> 2019-08-28 13:22:40,898 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com
>>> and
>>> associated storage pools for this VM
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.s.StorageManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking pool: 1 for storage allocation , maxSize :
>>> 984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
>>> disable threshold: 0.85
>>> 2019-08-28 13:22:40,899 DEBUG
>>> [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) ClusterScopeStoragePoolAllocator returning 1 suitable
>>> storage pools
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Trying to find a potenial host and associated storage
>>> pools from the suitable host/pool lists for this VM
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
>>> for volume: ROOT
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Returning Deployment Destination:
>>>
>>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> :
>>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
>>>
>>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> :
>>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Host: 1 can access pool: 1
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com
>>> and
>>> associated storage pools for this VM
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Returning Deployment Destination:
>>>
>>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> :
>>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
>>> 2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
>>>
>>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> :
>>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
>>> 2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) VM state transitted from :Starting to Starting with
>>> event:
>>> OperationRetryvm's original host id: null new host id: 1 host id before
>>> state transition: null
>>> 2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) VM state transitted from :Starting to Starting with
>>> event:
>>> OperationRetryvm's original host id: null new host id: 1 host id before
>>> state transition: null
>>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) We are allocating VM, increasing the used capacity of
>>> this
>>> host:1
>>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU:
>>> 500
>>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested
>>> RAM:
>>> 536870912
>>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0,
>>> old
>>> reserved: 0, actual total: 54000, total with overprovisioning: 216000;
>>> new
>>> used:500, reserved:0; requested cpu:500,alloc_from_last:false
>>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0,
>>> old
>>> reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
>>> requested mem: 536870912,alloc_from_last:false
>>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) We are allocating VM, increasing the used capacity of
>>> this
>>> host:1
>>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU:
>>> 500
>>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
>>> ,Requested RAM: 1073741824
>>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
>>> old reserved: 0, actual total: 54000, total with overprovisioning:
>>> 216000;
>>> new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
>>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
>>> 536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
>>> reserved: 0; requested mem: 1073741824,alloc_from_last:false
>>> 2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Network id=203 is already implemented
>>> 2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Network id=203 is already implemented
>>> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Asking VirtualRouter to prepare for
>>> Nic[184-67-null-137.74.82.208]
>>> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Asking VirtualRouter to prepare for
>>> Nic[187-68-null-137.74.82.209]
>>> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Asking SecurityGroupProvider to prepare for
>>> Nic[184-67-null-137.74.82.208]
>>> 2019-08-28 13:22:40,913 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Asking SecurityGroupProvider to prepare for
>>> Nic[187-68-null-137.74.82.209]
>>> 2019-08-28 13:22:40,914 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Network id=201 is already implemented
>>> 2019-08-28 13:22:40,915 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Network id=201 is already implemented
>>> 2019-08-28 13:22:40,919 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Network id=200 is already implemented
>>> 2019-08-28 13:22:40,920 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Network id=200 is already implemented
>>> 2019-08-28 13:22:40,922 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Releasing ip address for instance=186
>>> 2019-08-28 13:22:40,923 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Releasing ip address for instance=189
>>> 2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Allocated a nic
>>> NicProfile[186-67-c608b88c-7d06-411e-bb84-f2d5e2b3390d-192.168.0.226-null
>>> for VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Allocated a nic
>>> NicProfile[189-68-6c836a9c-0d16-42c3-86c7-446febb1a189-192.168.0.121-null
>>> for VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) template 3 is already in store:1, type:Image
>>> 2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) template 3 is already in store:1, type:Image
>>> 2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) template 3 is already in store:1, type:Primary
>>> 2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) template 3 is already in store:1, type:Primary
>>> 2019-08-28 13:22:40,932 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) copyAsync inspecting src type TEMPLATE copyAsync
>>> inspecting dest type VOLUME
>>> 2019-08-28 13:22:40,933 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) copyAsync inspecting src type TEMPLATE copyAsync
>>> inspecting dest type VOLUME
>>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) getCommandHostDelegation: class
>>> org.apache.cloudstack.storage.command.CopyCommand
>>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) We are returning the default host to execute commands
>>> because the target hypervisor of the source data is not XenServer.
>>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) getCommandHostDelegation: class
>>> org.apache.cloudstack.storage.command.CopyCommand
>>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) We are returning the default host to execute commands
>>> because the target hypervisor of the source data is not XenServer.
>>> 2019-08-28 13:22:40,934 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Seq 1-5778962746846478715: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
>>>
>>> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
>>> cloudstack.storage.to
>>> .TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
>>>
>>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
>>> ","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
>>> Template
>>> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
>>> }
>>> 2019-08-28 13:22:40,935 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Seq 1-5778962746846478716: Waiting for Seq
>>> 5778962746846478715 Scheduling: { Cmd , MgmtId: 205425461278937, via: 1(
>>> fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
>>>
>>> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
>>> cloudstack.storage.to
>>> .TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
>>>
>>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
>>> ","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
>>> Template
>>> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
>>> }
>>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-138d8f65) (logid:85352d35) ===START===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128974
>>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478715:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 110,
>>>
>>> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
>>> }
>>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.m.AgentAttache]
>>> (AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478716: Sending
>>> now. is current sequence.
>>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Seq 1-5778962746846478715: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
>>> CopyCmdAnswer } }
>>> 2019-08-28 13:22:41,040 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:41,047 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128974
>>> 2019-08-28 13:22:41,057 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
>>> template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
>>> zone=1 pod=1 guid=s-67-VM workers=5
>>>
>>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>>> dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
>>> 2019-08-28 13:22:41,071 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Seq 1-5778962746846478717: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>>
>>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>>> dns1=8.8.8.8 dns2=8.8.4.4
>>>
>>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
>>> }
>>> 2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 110,
>>>
>>> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
>>> }
>>> 2019-08-28 13:22:41,126 DEBUG [c.c.a.m.AgentAttache]
>>> (AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716: No more
>>> commands found
>>> 2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Seq 1-5778962746846478716: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
>>> CopyCmdAnswer } }
>>> 2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-9:null) (logid:) Seq 1-5778962746846478717:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>>
>>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>>
>>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>>> dns1=8.8.8.8 dns2=8.8.4.4
>>>
>>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>>> to create vnet 100: device eth1.100 is already a member of a bridge;
>>> can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>>
>>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>>> by previous failure","wait":0}}] }
>>> 2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Seq 1-5778962746846478717: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StartAnswer, Answer } }
>>> 2019-08-28 13:22:41,128 INFO [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
>>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>> breth1-100
>>> 2019-08-28 13:22:41,129 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Cleaning up resources for the vm
>>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>>> 2019-08-28 13:22:41,131 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Seq 1-5778962746846478718: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.64","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
>>> }
>>> 2019-08-28 13:22:41,139 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
>>> type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
>>> guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
>>> eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.2.191
>>> eth0mask=255.255.0.0 eth1ip=192.168.0.121 eth1mask=255.255.255.0
>>> mgmtcidr=
>>> 51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
>>> internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
>>> 2019-08-28 13:22:41,151 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Seq 1-5778962746846478719: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>>> port=8250
>>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68
>>> disable_rp_filter=true
>>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>>
>>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.191","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
>>> }
>>> 2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-8:null) (logid:) Seq 1-5778962746846478719:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>>
>>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>>> port=8250
>>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68
>>> disable_rp_filter=true
>>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>>
>>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>>> to create vnet 100: device eth1.100 is already a member of a bridge;
>>> can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>>
>>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>>> by previous failure","wait":0}}] }
>>> 2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Seq 1-5778962746846478719: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StartAnswer, Answer } }
>>> 2019-08-28 13:22:41,323 INFO [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
>>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>> breth1-100
>>> 2019-08-28 13:22:41,326 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Cleaning up resources for the vm
>>> VM[ConsoleProxy|v-68-VM]
>>> in Starting state
>>> 2019-08-28 13:22:41,337 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Seq 1-5778962746846478720: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.2.191","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
>>> }
>>> 2019-08-28 13:22:44,313 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-10:null) (logid:) Seq 1-5778962746846478718:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>>> 2019-08-28 13:22:44,314 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Seq 1-5778962746846478718: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StopAnswer } }
>>> 2019-08-28 13:22:44,318 DEBUG [c.c.n.g.ControlNetworkGuru]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
>>> 2019-08-28 13:22:44,321 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Releasing ip address for
>>> reservationId=c608b88c-7d06-411e-bb84-f2d5e2b3390d, instance=186
>>> 2019-08-28 13:22:44,321 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
>>> 2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Successfully released network resources for the vm
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Successfully cleaned up resources for the VM
>>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>>> 2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Root volume is ready, need to place VM in volume's
>>> cluster
>>> 2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
>>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>>> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
>>> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
>>> 2019-08-28 13:22:44,324 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
>>> 2019-08-28 13:22:44,324 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:22:44,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:22:44,327 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:22:44,328 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>>> Ram=512
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com
>>> ,
>>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
>>> suitable hosts
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) No suitable hosts found
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) No suitable hosts found under this Cluster: 1
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Could not find suitable Deployment Destination for this
>>> VM
>>> under any clusters, returning.
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) The specified cluster is in avoid set, returning.
>>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>>> pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
>>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
>>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
>>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Searching all possible resources under this Zone: 1
>>> 2019-08-28 13:22:44,332 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Listing clusters in order of aggregate capacity, that
>>> have
>>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>>> 2019-08-28 13:22:44,333 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Removing from the clusterId list these clusters from
>>> avoid
>>> set: [1]
>>> 2019-08-28 13:22:44,334 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) No clusters found after removing disabled clusters and
>>> clusters in avoid list, returning.
>>> 2019-08-28 13:22:44,338 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) VM state transitted from :Starting to Stopped with
>>> event:
>>> OperationFailedvm's original host id: null new host id: null host id
>>> before
>>> state transition: 1
>>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
>>> applying overprovisioning: 66088833024
>>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
>>> actual total: 54000, total with overprovisioning: 216000; new used:
>>> 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
>>> 0, total: 66088833024; new used: 1073741824,reserved:0;
>>> movedfromreserved:
>>> false,moveToReserveredfalse
>>> 2019-08-28 13:22:44,343 ERROR [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Invocation exception, caused by:
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:22:44,343 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>>> (logid:e3068c13) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:22:44,343 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin:
>>> 76
>>> 2019-08-28 13:22:44,344 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>>> Unable
>>> to complete AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType:
>>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>>> null}, job origin:76
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:22:44,344 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>>> Complete async job-218, jobStatus: FAILED, resultCode: 0, result:
>>>
>>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>>> Publish async job-218 complete on message bus
>>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
>>> up jobs related to job-218
>>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>>> Update
>>> db status for job-218
>>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
>>> up jobs joined with job-218 and disjoin all subjobs created from job- 218
>>> 2019-08-28 13:22:44,349 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
>>> executing com.cloud.vm.VmWorkStart for job-218
>>> 2019-08-28 13:22:44,350 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>>> Remove
>>> job-218 from job monitoring
>>> 2019-08-28 13:22:44,351 WARN [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Exception while trying to
>>> start secondary storage vm
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:22:44,352 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Unable to start secondary
>>> storage vm for standby capacity, vm id : 67, will recycle it and start a
>>> new one
>>> 2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage
>>> vm
>>> alert
>>> 2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Secondary Storage Vm
>>> creation
>>> failure, zone: France
>>> 2019-08-28 13:22:44,352 WARN [c.c.a.AlertManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) AlertType:: 19 |
>>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
>>> Storage Vm creation failure. zone: France, error details: null
>>> 2019-08-28 13:22:44,353 INFO
>>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Primary secondary storage is
>>> not even started, wait until next turn
>>> 2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-11:null) (logid:) Seq 1-5778962746846478720:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>>> 2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Seq 1-5778962746846478720: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StopAnswer } }
>>> 2019-08-28 13:22:44,436 DEBUG [c.c.n.g.ControlNetworkGuru]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
>>> 2019-08-28 13:22:44,438 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Releasing ip address for
>>> reservationId=6c836a9c-0d16-42c3-86c7-446febb1a189, instance=189
>>> 2019-08-28 13:22:44,438 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
>>> 2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Successfully released network resources for the vm
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Successfully cleaned up resources for the VM
>>> VM[ConsoleProxy|v-68-VM] in Starting state
>>> 2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Root volume is ready, need to place VM in volume's
>>> cluster
>>> 2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
>>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
>>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
>>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
>>> 2019-08-28 13:22:44,441 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:22:44,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:22:44,443 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>>> Ram=1024
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com
>>> ,
>>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
>>> suitable hosts
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) No suitable hosts found
>>> 2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) No suitable hosts found under this Cluster: 1
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Could not find suitable Deployment Destination for this
>>> VM
>>> under any clusters, returning.
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) The specified cluster is in avoid set, returning.
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>>> pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
>>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
>>> 2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Searching all possible resources under this Zone: 1
>>> 2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that
>>> have
>>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>>> 2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Removing from the clusterId list these clusters from
>>> avoid
>>> set: [1]
>>> 2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) No clusters found after removing disabled clusters and
>>> clusters in avoid list, returning.
>>> 2019-08-28 13:22:44,450 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) VM state transitted from :Starting to Stopped with
>>> event:
>>> OperationFailedvm's original host id: null new host id: null host id
>>> before
>>> state transition: 1
>>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
>>> applying overprovisioning: 66088833024
>>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
>>> actual total: 54000, total with overprovisioning: 216000; new used:
>>> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
>>> 0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
>>> false,moveToReserveredfalse
>>> 2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Invocation exception, caused by:
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:22:44,453 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>>> (logid:3e502c4b) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:22:44,453 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin:
>>> 77
>>> 2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>>> Unable
>>> to complete AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType:
>>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>>> null}, job origin:77
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>>> Complete async job-219, jobStatus: FAILED, resultCode: 0, result:
>>>
>>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>>> Publish async job-219 complete on message bus
>>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
>>> up jobs related to job-219
>>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>>> Update
>>> db status for job-219
>>> 2019-08-28 13:22:44,455 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
>>> up jobs joined with job-219 and disjoin all subjobs created from job- 219
>>> 2019-08-28 13:22:44,457 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
>>> executing com.cloud.vm.VmWorkStart for job-219
>>> 2019-08-28 13:22:44,458 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>>> Remove
>>> job-219 from job monitoring
>>> 2019-08-28 13:22:44,459 WARN [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Exception while trying to
>>> start console proxy
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:22:44,460 INFO [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Unable to start console
>>> proxy vm for standby capacity, vm id : 68, will recycle it and start a
>>> new
>>> one
>>> 2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy
>>> alert
>>> 2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Console proxy creation
>>> failure, zone: France
>>> 2019-08-28 13:22:44,460 WARN [c.c.a.AlertManagerImpl]
>>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) AlertType:: 10 |
>>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
>>> proxy creation failure. zone: France, error details: null
>>> 2019-08-28 13:22:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:22:44,852 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 routers to
>>> update status.
>>> 2019-08-28 13:22:44,854 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 VPC
>>> networks
>>> to update Redundant State.
>>> 2019-08-28 13:22:44,856 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 networks to
>>> update RvR status.
>>> 2019-08-28 13:22:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:48,876 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-1ca6b8a4) (logid:767cda38) ===START===
>>> 137.74.35.102
>>> -- GET command=listInfrastructure&response=json&_=1566988128975
>>> 2019-08-28 13:22:48,883 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:48,903 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128975
>>> 2019-08-28 13:22:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:22:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:22:56,741 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-3fa512ab) (logid:4609e172) ===START===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128976
>>> 2019-08-28 13:22:56,747 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:22:56,751 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128976
>>> 2019-08-28 13:23:03,063 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-a0c69e67) (logid:0a2b6c35) ===START===
>>> 137.74.35.102
>>> -- GET command=listInfrastructure&response=json&_=1566988128977
>>> 2019-08-28 13:23:03,071 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:23:03,091 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128977
>>> 2019-08-28 13:23:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:23:04,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:23:09,900 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-16:ctx-9f3738f2) (logid:dea95b1b) ===START===
>>> 137.74.35.102
>>> -- GET command=listInfrastructure&response=json&_=1566988128978
>>> 2019-08-28 13:23:09,907 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:23:09,929 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128978
>>> 2019-08-28 13:23:10,557 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Zone 1 is ready to launch
>>> secondary storage VM
>>> 2019-08-28 13:23:10,561 INFO
>>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) No running secondary storage
>>> vms found in datacenter id=1, starting one
>>> 2019-08-28 13:23:10,570 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Found a stopped secondary
>>> storage vm, starting it. Vm id : 67
>>> 2019-08-28 13:23:10,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-220 execution on
>>> object VmWorkJobQueue.67
>>> 2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Zone 1 is ready to launch
>>> console proxy
>>> 2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expand console proxy
>>> standby
>>> capacity for zone France
>>> 2019-08-28 13:23:10,616 INFO [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Found a stopped console
>>> proxy, starting it. Vm id : 68
>>> 2019-08-28 13:23:10,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-221 execution on
>>> object VmWorkJobQueue.68
>>> 2019-08-28 13:23:10,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute
>>> sync-queue
>>> item: SyncQueueItemVO {id:196, queueId: 191, contentType: AsyncJob,
>>> contentId: 220, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
>>> lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
>>> 13:23:10 CEST 2019}
>>> 2019-08-28 13:23:10,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
>>> job-220
>>> 2019-08-28 13:23:10,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute
>>> sync-queue
>>> item: SyncQueueItemVO {id:197, queueId: 192, contentType: AsyncJob,
>>> contentId: 221, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
>>> lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
>>> 13:23:10 CEST 2019}
>>> 2019-08-28 13:23:10,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
>>> job-221
>>> 2019-08-28 13:23:10,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:f78794fa) Add
>>> job-220 into job monitoring
>>> 2019-08-28 13:23:10,871 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:7deee6cf) Add
>>> job-221 into job monitoring
>>> 2019-08-28 13:23:10,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>>> Executing AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType:
>>> null,
>>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>>> null}
>>> 2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Run
>>> VM
>>> work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
>>> 2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Execute VM work job:
>>>
>>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:23:10,881 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>>> Executing AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType:
>>> null,
>>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>>> null}
>>> 2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Run
>>> VM
>>> work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
>>> 2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Execute VM work job:
>>>
>>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:23:10,883 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) VM state transitted from :Stopped to Starting with
>>> event:
>>> StartRequestedvm's original host id: null new host id: null host id
>>> before
>>> state transition: null
>>> 2019-08-28 13:23:10,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Successfully transitioned to start state for
>>> VM[SecondaryStorageVm|s-67-VM] reservation id =
>>> 46a0e954-a9af-45af-ac39-91e036654f95
>>> 2019-08-28 13:23:10,884 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) VM state transitted from :Stopped to Starting with
>>> event:
>>> StartRequestedvm's original host id: null new host id: null host id
>>> before
>>> state transition: null
>>> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Successfully transitioned to start state for
>>> VM[ConsoleProxy|v-68-VM] reservation id =
>>> a25c0118-b49c-41b2-8970-9b73191746f3
>>> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Root volume is ready, need to place VM in volume's
>>> cluster
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
>>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Root volume is ready, need to place VM in volume's
>>> cluster
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
>>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>>> 2019-08-28 13:23:10,886 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>>> Ram=1024
>>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>>> Ram=512
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
>>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
>>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
>>> capacity for requested CPU: 500 and requested RAM: 1073741824 ,
>>> cpuOverprovisioningFactor: 4.0
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
>>> capacity for requested CPU: 500 and requested RAM: 536870912 ,
>>> cpuOverprovisioningFactor: 4.0
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU:
>>> 54000
>>> and CPU after applying overprovisioning: 216000
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
>>> CPU: 500
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
>>> Requested RAM: 1073741824
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
>>> available
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from
>>> host:
>>> 1, used: 0, reserved: 0, actual total: 54000, total with
>>> overprovisioning:
>>> 216000; requested cpu:500,alloc_from_last_host?:false
>>> ,considerReservedCapacity?: true
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from
>>> host:
>>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>>> 1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding
>>> to
>>> list: 1
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
>>> suitable hosts
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU:
>>> 54000
>>> and CPU after applying overprovisioning: 216000
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
>>> CPU: 500
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
>>> Requested RAM: 536870912
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
>>> available
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from
>>> host:
>>> 1, used: 0, reserved: 0, actual total: 54000, total with
>>> overprovisioning:
>>> 216000; requested cpu:500,alloc_from_last_host?:false
>>> ,considerReservedCapacity?: true
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from
>>> host:
>>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>>> 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding
>>> to
>>> list: 1
>>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
>>> suitable hosts
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Volume has pool already allocated, checking if pool can
>>> be
>>> reused, poolId: 1
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Volume has pool already allocated, checking if pool can
>>> be
>>> reused, poolId: 1
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Planner need not allocate a pool for this volume since
>>> its
>>> READY
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Trying to find a potenial host and associated storage
>>> pools from the suitable host/pool lists for this VM
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
>>> for volume: ROOT
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Planner need not allocate a pool for this volume since
>>> its
>>> READY
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Trying to find a potenial host and associated storage
>>> pools from the suitable host/pool lists for this VM
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Checking if host: 1 can access any suitable storage pool
>>> for volume: ROOT
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Host: 1 can access pool: 1
>>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Host: 1 can access pool: 1
>>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com
>>> and
>>> associated storage pools for this VM
>>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com
>>> and
>>> associated storage pools for this VM
>>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Returning Deployment Destination:
>>>
>>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>>> 2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
>>>
>>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Returning Deployment Destination:
>>>
>>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>>> 2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
>>>
>>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>>> 2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) VM state transitted from :Starting to Starting with
>>> event:
>>> OperationRetryvm's original host id: null new host id: 1 host id before
>>> state transition: null
>>> 2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) VM state transitted from :Starting to Starting with
>>> event:
>>> OperationRetryvm's original host id: null new host id: 1 host id before
>>> state transition: null
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) We are allocating VM, increasing the used capacity of
>>> this
>>> host:1
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU:
>>> 500
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested
>>> RAM:
>>> 536870912
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0,
>>> old
>>> reserved: 0, actual total: 54000, total with overprovisioning: 216000;
>>> new
>>> used:500, reserved:0; requested cpu:500,alloc_from_last:false
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0,
>>> old
>>> reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
>>> requested mem: 536870912,alloc_from_last:false
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) We are allocating VM, increasing the used capacity of
>>> this
>>> host:1
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU:
>>> 500
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
>>> ,Requested RAM: 1073741824
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
>>> old reserved: 0, actual total: 54000, total with overprovisioning:
>>> 216000;
>>> new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
>>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
>>> 536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
>>> reserved: 0; requested mem: 1073741824,alloc_from_last:false
>>> 2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Network id=203 is already implemented
>>> 2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Network id=203 is already implemented
>>> 2019-08-28 13:23:10,907 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Asking VirtualRouter to prepare for
>>> Nic[187-68-null-137.74.82.209]
>>> 2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Asking VirtualRouter to prepare for
>>> Nic[184-67-null-137.74.82.208]
>>> 2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Asking SecurityGroupProvider to prepare for
>>> Nic[187-68-null-137.74.82.209]
>>> 2019-08-28 13:23:10,909 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Asking SecurityGroupProvider to prepare for
>>> Nic[184-67-null-137.74.82.208]
>>> 2019-08-28 13:23:10,911 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Network id=201 is already implemented
>>> 2019-08-28 13:23:10,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Network id=201 is already implemented
>>> 2019-08-28 13:23:10,916 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Network id=200 is already implemented
>>> 2019-08-28 13:23:10,918 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Network id=200 is already implemented
>>> 2019-08-28 13:23:10,918 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Releasing ip address for instance=186
>>> 2019-08-28 13:23:10,920 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Allocated a nic
>>> NicProfile[186-67-46a0e954-a9af-45af-ac39-91e036654f95-192.168.0.86-null
>>> for VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:23:10,921 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Releasing ip address for instance=189
>>> 2019-08-28 13:23:10,922 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) No need to recreate the volume: Vol[62|vm=67|ROOT],
>>> since
>>> it already has a pool assigned: 1, adding disk to VM
>>> 2019-08-28 13:23:10,923 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Allocated a nic
>>> NicProfile[189-68-a25c0118-b49c-41b2-8970-9b73191746f3-192.168.0.20-null
>>> for VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:23:10,925 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) No need to recreate the volume: Vol[63|vm=68|ROOT],
>>> since
>>> it already has a pool assigned: 1, adding disk to VM
>>> 2019-08-28 13:23:10,930 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
>>> template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
>>> zone=1 pod=1 guid=s-67-VM workers=5
>>>
>>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>>> dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
>>> 2019-08-28 13:23:10,931 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
>>> type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
>>> guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
>>> eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.0.248
>>> eth0mask=255.255.0.0 eth1ip=192.168.0.20 eth1mask=255.255.255.0 mgmtcidr=
>>> 51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
>>> internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
>>> 2019-08-28 13:23:10,941 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Seq 1-5778962746846478721: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>>
>>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>>> dns1=8.8.8.8 dns2=8.8.4.4
>>>
>>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.170","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
>>> }
>>> 2019-08-28 13:23:10,942 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Seq 1-5778962746846478722: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>>> port=8250
>>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68
>>> disable_rp_filter=true
>>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>>
>>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.248","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
>>> }
>>> 2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-12:null) (logid:) Seq 1-5778962746846478722:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>>
>>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>>> port=8250
>>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68
>>> disable_rp_filter=true
>>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>>
>>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>>> to create vnet 100: device eth1.100 is already a member of a bridge;
>>> can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>>
>>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>>> by previous failure","wait":0}}] }
>>> 2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Seq 1-5778962746846478722: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StartAnswer, Answer } }
>>> 2019-08-28 13:23:10,985 INFO [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
>>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>> breth1-100
>>> 2019-08-28 13:23:10,986 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Cleaning up resources for the vm
>>> VM[ConsoleProxy|v-68-VM]
>>> in Starting state
>>> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Seq 1-5778962746846478723: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.0.248","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
>>> }
>>> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-13:null) (logid:) Seq 1-5778962746846478721:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>>
>>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>>
>>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>>> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
>>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>>> dns1=8.8.8.8 dns2=8.8.4.4
>>>
>>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>>> to create vnet 100: device eth1.100 is already a member of a bridge;
>>> can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>>
>>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>>> by previous failure","wait":0}}] }
>>> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Seq 1-5778962746846478721: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StartAnswer, Answer } }
>>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
>>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>> breth1-100
>>> 2019-08-28 13:23:10,990 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Cleaning up resources for the vm
>>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>>> 2019-08-28 13:23:10,991 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Seq 1-5778962746846478724: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.170","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
>>> }
>>> 2019-08-28 13:23:14,130 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-14:null) (logid:) Seq 1-5778962746846478724:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>>> 2019-08-28 13:23:14,131 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Seq 1-5778962746846478724: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StopAnswer } }
>>> 2019-08-28 13:23:14,136 DEBUG [c.c.n.g.ControlNetworkGuru]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
>>> 2019-08-28 13:23:14,138 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Releasing ip address for
>>> reservationId=46a0e954-a9af-45af-ac39-91e036654f95, instance=186
>>> 2019-08-28 13:23:14,139 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
>>> 2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Successfully released network resources for the vm
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Successfully cleaned up resources for the VM
>>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>>> 2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Root volume is ready, need to place VM in volume's
>>> cluster
>>> 2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
>>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
>>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
>>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
>>> 2019-08-28 13:23:14,142 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:23:14,145 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:23:14,145 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:23:14,146 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>>> Ram=512
>>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com
>>> ,
>>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
>>> suitable hosts
>>> 2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) No suitable hosts found
>>> 2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) No suitable hosts found under this Cluster: 1
>>> 2019-08-28 13:23:14,148 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Could not find suitable Deployment Destination for this
>>> VM
>>> under any clusters, returning.
>>> 2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) The specified cluster is in avoid set, returning.
>>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:null, requested cpu: 500, requested ram: 536870912
>>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
>>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
>>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Searching resources only under specified Pod: 1
>>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Listing clusters in order of aggregate capacity, that
>>> have
>>> (atleast one host with) enough CPU and RAM capacity under this Pod: 1
>>> 2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Removing from the clusterId list these clusters from
>>> avoid
>>> set: [1]
>>> 2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) No clusters found after removing disabled clusters and
>>> clusters in avoid list, returning.
>>> 2019-08-28 13:23:14,156 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) VM state transitted from :Starting to Stopped with
>>> event:
>>> OperationFailedvm's original host id: null new host id: null host id
>>> before
>>> state transition: 1
>>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
>>> applying overprovisioning: 66088833024
>>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
>>> actual total: 54000, total with overprovisioning: 216000; new used:
>>> 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
>>> 0, total: 66088833024; new used: 1073741824,reserved:0;
>>> movedfromreserved:
>>> false,moveToReserveredfalse
>>> 2019-08-28 13:23:14,160 ERROR [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Invocation exception, caused by:
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:23:14,161 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>>> (logid:e3068c13) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:23:14,161 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin:
>>> 76
>>> 2019-08-28 13:23:14,161 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>>> Unable
>>> to complete AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType:
>>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>>> null}, job origin:76
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:23:14,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>>> Complete async job-220, jobStatus: FAILED, resultCode: 0, result:
>>>
>>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>>> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>>> Publish async job-220 complete on message bus
>>> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
>>> up jobs related to job-220
>>> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>>> Update
>>> db status for job-220
>>> 2019-08-28 13:23:14,163 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
>>> up jobs joined with job-220 and disjoin all subjobs created from job- 220
>>> 2019-08-28 13:23:14,166 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
>>> executing com.cloud.vm.VmWorkStart for job-220
>>> 2019-08-28 13:23:14,166 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>>> Remove
>>> job-220 from job monitoring
>>> 2019-08-28 13:23:14,169 WARN [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Exception while trying to
>>> start secondary storage vm
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:23:14,169 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Unable to start secondary
>>> storage vm for standby capacity, vm id : 67, will recycle it and start a
>>> new one
>>> 2019-08-28 13:23:14,172 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-222 execution on
>>> object VmWorkJobQueue.67
>>> 2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-15:null) (logid:) Seq 1-5778962746846478723:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>>> 2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Seq 1-5778962746846478723: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> StopAnswer } }
>>> 2019-08-28 13:23:14,623 DEBUG [c.c.n.g.ControlNetworkGuru]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
>>> 2019-08-28 13:23:14,626 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Releasing ip address for
>>> reservationId=a25c0118-b49c-41b2-8970-9b73191746f3, instance=189
>>> 2019-08-28 13:23:14,626 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
>>> 2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Successfully released network resources for the vm
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Successfully cleaned up resources for the VM
>>> VM[ConsoleProxy|v-68-VM] in Starting state
>>> 2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Root volume is ready, need to place VM in volume's
>>> cluster
>>> 2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
>>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
>>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
>>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
>>> 2019-08-28 13:23:14,631 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:23:14,635 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>>> 2019-08-28 13:23:14,635 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>>> pod:1 cluster:1
>>> 2019-08-28 13:23:14,636 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>>> to
>>> check for allocation: [Host[-1-Routing]]
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>>> after prioritization: [Host[-1-Routing]]
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>>> Ram=1024
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com
>>> ,
>>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
>>> suitable hosts
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) No suitable hosts found
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) No suitable hosts found under this Cluster: 1
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Could not find suitable Deployment Destination for this
>>> VM
>>> under any clusters, returning.
>>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) The specified cluster is in avoid set, returning.
>>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>>> pod:1,cluster:null, requested cpu: 500, requested ram: 1073741824
>>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
>>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
>>> 2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Searching resources only under specified Pod: 1
>>> 2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that
>>> have
>>> (atleast one host with) enough CPU and RAM capacity under this Pod: 1
>>> 2019-08-28 13:23:14,641 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Removing from the clusterId list these clusters from
>>> avoid
>>> set: [1]
>>> 2019-08-28 13:23:14,642 DEBUG [c.c.d.FirstFitPlanner]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) No clusters found after removing disabled clusters and
>>> clusters in avoid list, returning.
>>> 2019-08-28 13:23:14,646 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) VM state transitted from :Starting to Stopped with
>>> event:
>>> OperationFailedvm's original host id: null new host id: null host id
>>> before
>>> state transition: 1
>>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>>> overprovisioning: 216000
>>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
>>> applying overprovisioning: 66088833024
>>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
>>> actual total: 54000, total with overprovisioning: 216000; new used:
>>> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
>>> 0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
>>> false,moveToReserveredfalse
>>> 2019-08-28 13:23:14,650 ERROR [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Invocation exception, caused by:
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:23:14,651 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>>> (logid:3e502c4b) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> 2019-08-28 13:23:14,651 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin:
>>> 77
>>> 2019-08-28 13:23:14,651 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>>> Unable
>>> to complete AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType:
>>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>>
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>>> null}, job origin:77
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>>> Complete async job-221, jobStatus: FAILED, resultCode: 0, result:
>>>
>>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>>> Publish async job-221 complete on message bus
>>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
>>> up jobs related to job-221
>>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>>> Update
>>> db status for job-221
>>> 2019-08-28 13:23:14,653 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
>>> up jobs joined with job-221 and disjoin all subjobs created from job- 221
>>> 2019-08-28 13:23:14,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
>>> executing com.cloud.vm.VmWorkStart for job-221
>>> 2019-08-28 13:23:14,656 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>>> Remove
>>> job-221 from job monitoring
>>> 2019-08-28 13:23:14,658 WARN [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Exception while trying to
>>> start console proxy
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>> create a
>>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>>> com.cloud.dc.DataCenter; id=1
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>>> at
>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> 2019-08-28 13:23:14,658 INFO [c.c.c.ConsoleProxyManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Unable to start console
>>> proxy vm for standby capacity, vm id : 68, will recycle it and start a
>>> new
>>> one
>>> 2019-08-28 13:23:14,662 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-223 execution on
>>> object VmWorkJobQueue.68
>>> 2019-08-28 13:23:14,852 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 routers to
>>> update status.
>>> 2019-08-28 13:23:14,854 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 VPC
>>> networks
>>> to update Redundant State.
>>> 2019-08-28 13:23:14,855 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute
>>> sync-queue
>>> item: SyncQueueItemVO {id:198, queueId: 191, contentType: AsyncJob,
>>> contentId: 222, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
>>> lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
>>> 13:23:14 CEST 2019}
>>> 2019-08-28 13:23:14,856 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 networks to
>>> update RvR status.
>>> 2019-08-28 13:23:14,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
>>> job-222
>>> 2019-08-28 13:23:14,859 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute
>>> sync-queue
>>> item: SyncQueueItemVO {id:199, queueId: 192, contentType: AsyncJob,
>>> contentId: 223, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
>>> lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
>>> 13:23:14 CEST 2019}
>>> 2019-08-28 13:23:14,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
>>> job-223
>>> 2019-08-28 13:23:14,863 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:7e4a2784) Add
>>> job-222 into job monitoring
>>> 2019-08-28 13:23:14,869 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:26933ca4) Add
>>> job-223 into job monitoring
>>> 2019-08-28 13:23:14,871 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:23:14,875 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
>>> Executing AsyncJobVO {id:222, userId: 1, accountId: 1, instanceType:
>>> null,
>>> instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
>>>
>>> rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
>>> null}
>>> 2019-08-28 13:23:14,875 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Run
>>> VM
>>> work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
>>> 2019-08-28 13:23:14,877 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Execute VM work job:
>>>
>>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:23:14,879 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:23:14,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
>>> Executing AsyncJobVO {id:223, userId: 1, accountId: 1, instanceType:
>>> null,
>>> instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
>>>
>>> rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
>>> null}
>>> 2019-08-28 13:23:14,881 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Run
>>> VM
>>> work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
>>> 2019-08-28 13:23:14,882 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Execute VM work job:
>>>
>>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:23:14,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) VM is already stopped: VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:23:14,883 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Done executing VM work job:
>>>
>>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:23:14,883 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Complete async job-222, jobStatus: SUCCEEDED,
>>> resultCode:
>>> 0, result: null
>>> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Publish async job-222 complete on message bus
>>> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Wake up jobs related to job-222
>>> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Update db status for job-222
>>> 2019-08-28 13:23:14,886 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>>> (logid:e3068c13) Wake up jobs joined with job-222 and disjoin all subjobs
>>> created from job- 222
>>> 2019-08-28 13:23:14,888 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) VM is already stopped: VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:23:14,888 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Done executing VM work job:
>>>
>>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>>> 2019-08-28 13:23:14,888 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Complete async job-223, jobStatus: SUCCEEDED,
>>> resultCode:
>>> 0, result: null
>>> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Publish async job-223 complete on message bus
>>> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Wake up jobs related to job-223
>>> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Update db status for job-223
>>> 2019-08-28 13:23:14,892 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>>> (logid:3e502c4b) Wake up jobs joined with job-223 and disjoin all subjobs
>>> created from job- 223
>>> 2019-08-28 13:23:14,895 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStop for VM 67, job origin:
>>> 76
>>> 2019-08-28 13:23:14,895 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
>>> executing com.cloud.vm.VmWorkStop for job-222
>>> 2019-08-28 13:23:14,898 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
>>> Remove
>>> job-222 from job monitoring
>>> 2019-08-28 13:23:14,901 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStop for VM 68, job origin:
>>> 77
>>> 2019-08-28 13:23:14,901 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
>>> executing com.cloud.vm.VmWorkStop for job-223
>>> 2019-08-28 13:23:14,903 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
>>> Remove
>>> job-223 from job monitoring
>>> 2019-08-28 13:23:14,916 DEBUG [c.c.c.CapacityManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) VM state transitted from
>>> :Stopped to Expunging with event: ExpungeOperationvm's original host id:
>>> null new host id: null host id before state transition: null
>>> 2019-08-28 13:23:14,921 DEBUG [c.c.c.CapacityManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) VM state transitted from
>>> :Stopped to Expunging with event: ExpungeOperationvm's original host id:
>>> null new host id: null host id before state transition: null
>>> 2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Destroying vm
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up NICS
>>> 2019-08-28 13:23:14,923 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning network for vm: 67
>>> 2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Destroying vm
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up NICS
>>> 2019-08-28 13:23:14,929 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning network for vm:
>>> 68
>>> 2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking VirtualRouter to
>>> release Nic[184-67-null-137.74.82.208]
>>> 2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking SecurityGroupProvider
>>> to release Nic[184-67-null-137.74.82.208]
>>> 2019-08-28 13:23:14,936 DEBUG [c.c.n.g.DirectNetworkGuru]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Deallocate network:
>>> networkId:
>>> 203, ip: 137.74.82.208
>>> 2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking VirtualRouter to
>>> release Nic[187-68-null-137.74.82.209]
>>> 2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking
>>> SecurityGroupProvider
>>> to release Nic[187-68-null-137.74.82.209]
>>> 2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Deallocate network:
>>> networkId: 203, ip: 137.74.82.209
>>> 2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) remove nic 184 secondary ip
>>> 2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=184
>>> 2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
>>> entry ...
>>> 2019-08-28 13:23:14,943 DEBUG [c.c.n.g.DirectNetworkGuru]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) remove nic 187 secondary
>>> ip
>>> 2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=187
>>> 2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
>>> entry ...
>>> 2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=185
>>> 2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
>>> entry ...
>>> 2019-08-28 13:23:14,949 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=188
>>> 2019-08-28 13:23:14,950 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
>>> entry ...
>>> 2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=186
>>> 2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
>>> entry ...
>>> 2019-08-28 13:23:14,952 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up hypervisor data
>>> structures (ex. SRs in XenServer) for managed storage
>>> 2019-08-28 13:23:14,952 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning storage for vm: 67
>>> 2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=189
>>> 2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
>>> entry ...
>>> 2019-08-28 13:23:14,953 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up hypervisor
>>> data
>>> structures (ex. SRs in XenServer) for managed storage
>>> 2019-08-28 13:23:14,954 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning storage for vm:
>>> 68
>>> 2019-08-28 13:23:14,964 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) getCommandHostDelegation:
>>> class org.apache.cloudstack.storage.command.DeleteCommand
>>> 2019-08-28 13:23:14,964 DEBUG [c.c.h.XenServerGuru]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) We are returning the default
>>> host to execute commands because the command is not of Copy type.
>>> 2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
>>> (secstorage-1:ctx-59a5da7a)
>>> (logid:2183e74c) Seq 1-5778962746846478725: Sending { Cmd , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>>
>>> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
>>> }
>>> 2019-08-28 13:23:14,965 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) getCommandHostDelegation:
>>> class org.apache.cloudstack.storage.command.DeleteCommand
>>> 2019-08-28 13:23:14,965 DEBUG [c.c.h.XenServerGuru]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) We are returning the
>>> default
>>> host to execute commands because the command is not of Copy type.
>>> 2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
>>> Sending { Cmd , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>>> Ver:
>>> v1, Flags: 100011,
>>>
>>> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.
>>> cloudstack.storage.to
>>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>>> apache.cloudstack.storage.to
>>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
>>> }
>>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-2:null) (logid:) Seq 1-5778962746846478726:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>>> (AgentManager-Handler-1:null) (logid:) Seq 1-5778962746846478725:
>>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>>> 10,
>>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
>>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>>> Ver: v1, Flags: 10, { Answer } }
>>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>>> (secstorage-1:ctx-59a5da7a)
>>> (logid:2183e74c) Seq 1-5778962746846478725: Received: { Ans: , MgmtId:
>>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>>> Answer }
>>> }
>>> 2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Volume 63 is not referred
>>> anywhere, remove it from volumes table
>>> 2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Volume 62 is not referred
>>> anywhere, remove it from volumes table
>>> 2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expunged
>>> VM[ConsoleProxy|v-68-VM]
>>> 2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Expunged
>>> VM[SecondaryStorageVm|s-67-VM]
>>> 2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) received secondary storage
>>> vm
>>> alert
>>> 2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Secondary Storage Vm
>>> creation
>>> failure, zone: France
>>> 2019-08-28 13:23:14,975 WARN [c.c.a.AlertManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) AlertType:: 19 |
>>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
>>> Storage Vm creation failure. zone: France, error details: null
>>> 2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) received console proxy
>>> alert
>>> 2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Console proxy creation
>>> failure, zone: France
>>> 2019-08-28 13:23:14,976 WARN [c.c.a.AlertManagerImpl]
>>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) AlertType:: 10 |
>>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
>>> proxy creation failure. zone: France, error details: null
>>> 2019-08-28 13:23:14,977 INFO
>>> [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Primary secondary storage is
>>> not even started, wait until next turn
>>> 2019-08-28 13:23:16,259 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-bd2c1e74) (logid:dc193f19) ===START===
>>> 137.74.35.102
>>> -- GET command=listInfrastructure&response=json&_=1566988128979
>>> 2019-08-28 13:23:16,268 DEBUG [c.c.a.ApiServer]
>>> (qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) CIDRs from
>>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>>> allowed
>>> to perform API calls: 0.0.0.0/0,::/0
>>> 2019-08-28 13:23:16,286 DEBUG [c.c.a.ApiServlet]
>>> (qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) ===END===
>>> 137.74.35.102 -- GET
>>> command=listInfrastructure&response=json&_=1566988128979
>>> 2019-08-28 13:23:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) Begin cleanup
>>> expired async-jobs
>>> 2019-08-28 13:23:24,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) End cleanup
>>> expired
>>> async-jobs
>>> 2019-08-28 13:23:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
>>> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>>> 2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>>> (AgentManager-Handler-3:null) (logid:) Process host VM state report from
>>> ping process. host: 1
>>> 2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>>> (AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
>>> number of records in report: 0
>>> 2019-08-28 13:23:24,958 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
>>> report. host: 1
>>>
>>
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
Any idea?
On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan <md...@gmail.com>
wrote:
> Thanks. I changed VLAN ID and now they are kept starting for along time.
> Latest logs:
>
>
> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin cleanup
> expired async-jobs
> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup expired
> async-jobs
> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been pending for
> 959 seconds
> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been pending for
> 357 seconds
> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin cleanup
> expired async-jobs
> 2019-08-28 14:34:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 routers to
> update status.
> 2019-08-28 14:34:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 14:34:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 networks to
> update RvR status.
> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup expired
> async-jobs
> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598229
> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598229
> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-e1f47dc0) (logid:c0c93cd1) Resetting hosts suitable for reconnect
> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed resetting hosts suitable for
> reconnect
> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) ===END===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566995598230
> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin cleanup
> expired async-jobs
> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup expired
> async-jobs
> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-15:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> 2019-08-28 14:34:24,974 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) Process host VM state report from
> ping process. host: 1
> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) Process VM state report. host: 1,
> number of records in report: 1
> 2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1, vm id:
> 103, power state: PowerOn
> 2019-08-28 14:34:24,976 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
> 1, vm id: 103, power state: PowerOn
> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 103, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) Run missing VM report. current
> time: 1566995664980
> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
> id: 101, power state: PowerReportMissing, last state update: 1566995544000
> 2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since last state
> update(120980ms) has passed graceful period
> 2019-08-28 14:34:24,982 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
> 1, vm id: 101, power state: PowerReportMissing
> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 101, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
> id: 102, power state: PowerReportMissing, last state update: 1566995544000
> 2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since last state
> update(120980ms) has passed graceful period
> 2019-08-28 14:34:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
> 1, vm id: 102, power state: PowerReportMissing
> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 102, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:34:24,987 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-15:null) (logid:) Done with process of VM state
> report. host: 1
> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598231
> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598231
> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) ===END===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566995598232
> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling Monitor is
> running...
> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) ===END===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) ===END===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
> Discovery plugin was unable to find an api by that name or process any apis
> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) ===END===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598235
> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin cleanup
> expired async-jobs
> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup expired
> async-jobs
> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
> 137.74.35.102 -- GET
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) ===END===
> 137.74.35.102 -- GET
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector is
> running...
> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq 1-5778962746846479041:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector is
> running...
> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no secondary
> storage VM for secondary storage host Secondary1
> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) getCommandHostDelegation:
> class com.cloud.agent.api.GetStorageStatsCommand
> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning the
> default host to execute commands because the command is not of Copy type.
> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq 1-5778962746846479042:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) ===END===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) ===END===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
> Discovery plugin was unable to find an api by that name or process any apis
> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) ===END===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598239
> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598240
> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598240
> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin cleanup
> expired async-jobs
> 2019-08-28 14:34:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 routers to
> update status.
> 2019-08-28 14:34:44,855 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 14:34:44,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 networks to
> update RvR status.
> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup expired
> async-jobs
> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:48,215 WARN [c.c.a.d.ParamGenericValidationWorker]
> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) Received
> unknown parameters for command listSystemVms. Unknown parameters : listall
> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
> 137.74.35.102 -- GET
> command=listHosts&response=json&details=min&_=1566995598242
> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) >>>Searching
> for hosts>>>
> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
> >>>Generating Response>>>
> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) ===END===
> 137.74.35.102 -- GET
> command=listHosts&response=json&details=min&_=1566995598242
> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin cleanup
> expired async-jobs
> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup expired
> async-jobs
> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) ===END===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598245
> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598245
> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin cleanup
> expired async-jobs
> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup expired
> async-jobs
> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) ===END===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566995598246
> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) ===END===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) ===END===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
> Discovery plugin was unable to find an api by that name or process any apis
> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) ===END===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598249
> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been pending for
> 1019 seconds
> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been pending for
> 417 seconds
> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
> 137.74.35.102 -- GET
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) ===END===
> 137.74.35.102 -- GET
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started resource
> counters recalculation periodic task.
> 2019-08-28 14:35:14,848 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External devices
> are not used. Skipping external device usage collection
> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin cleanup
> expired async-jobs
> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot scheduler.poll is
> being called at 2019-08-28 12:35:14 GMT
> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup expired
> async-jobs
> 2019-08-28 14:35:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 routers to
> update status.
> 2019-08-28 14:35:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterMonitor-1:ctx-7cb475c8) (logid:cc3e421d) Found 0 running routers.
> 2019-08-28 14:35:14,858 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to be
> executed at 2019-08-28 12:35:14 GMT
> 2019-08-28 14:35:14,858 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 networks to
> update RvR status.
> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4) NetworkGarbageCollector
> uses '600' seconds for GC interval.
> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage server
> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true, heartbeat:
> Wed Aug 28 14:35:12 CEST 2019
> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) ===END===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) ===END===
> 137.74.35.102 -- GET
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
> Discovery plugin was unable to find an api by that name or process any apis
> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) ===END===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566995598253
> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598254
> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566995598254
> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin cleanup
> expired async-jobs
> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup expired
> async-jobs
> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager is
> running and getting the updates from LB providers and updating service
> status
> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager is
> running and getting the updates from LB providers and updating service
> status
> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-1:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> 2019-08-28 14:35:25,005 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) Process host VM state report from
> ping process. host: 1
> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) Process VM state report. host: 1,
> number of records in report: 1
> 2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm id:
> 103, power state: PowerOn
> 2019-08-28 14:35:25,007 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) VM state report is updated. host: 1,
> vm id: 103, power state: PowerOn
> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-1:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 103, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:35:25,011 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) Run missing VM report. current time:
> 1566995725011
> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm id:
> 101, power state: PowerReportMissing, last state update: 1566995664000
> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last state
> update(61011ms) has not passed graceful period yet
> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm id:
> 102, power state: PowerReportMissing, last state update: 1566995664000
> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last state
> update(61011ms) has not passed graceful period yet
> 2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-1:null) (logid:) Done with process of VM state
> report. host: 1
> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:26,390 WARN [c.c.a.d.ParamGenericValidationWorker]
> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) Received
> unknown parameters for command listSystemVms. Unknown parameters : listall
> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
> 137.74.35.102 -- GET
> command=listHosts&response=json&details=min&_=1566995598256
> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) >>>Searching
> for hosts>>>
> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
> >>>Generating Response>>>
> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) ===END===
> 137.74.35.102 -- GET
> command=listHosts&response=json&details=min&_=1566995598256
> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq 1-5778962746846479043:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from executing
> class com.cloud.agent.api.GetVolumeStatsCommand:
> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling Monitor is
> running...
> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) ===END===
> 137.74.35.102 -- GET
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin cleanup
> expired async-jobs
> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup expired
> async-jobs
> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector is
> running...
> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq 1-5778962746846479044:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector is
> running...
> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no secondary
> storage VM for secondary storage host Secondary1
> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) getCommandHostDelegation:
> class com.cloud.agent.api.GetStorageStatsCommand
> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning the
> default host to execute commands because the command is not of Copy type.
> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq 1-5778962746846479045:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
> 2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Checking if
> any host reservation can be released ...
> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin cleanup
> expired async-jobs
> 2019-08-28 14:35:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 routers to
> update status.
> 2019-08-28 14:35:44,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 14:35:44,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 networks to
> update RvR status.
> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Cannot
> release reservation, Found 3 VMs Running on host 1
> 2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done running
> HostReservationReleaseChecker ...
> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup expired
> async-jobs
> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity Checker
> ...
> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating system
> capacity
> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram capacity
> update
> 2019-08-28 14:35:44,896 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 3 VMs on host 1
> 2019-08-28 14:35:44,904 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 0 VM, not running on
> host 1
> 2019-08-28 14:35:44,907 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
> capacity, host:1 usedCpuCore: 3 reservedCpuCore: 0
> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
> capacity, host:1 usedCpu: 1500 reservedCpu: 0
> 2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate memory
> capacity, host:1 usedMem: 1879048192 reservedMem: 0
> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing cpu/ram
> capacity update
> 2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing storage capacity
> update
> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Found storage pool Primary1
> of type SharedMountPoint
> 2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Total over provisioned
> capacity of the pool Primary1 id: 1 is 984895062016
> 2019-08-28 14:35:44,920 DEBUG [c.c.s.StorageManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Successfully set Capacity -
> 984895062016 for capacity type - 3 , DataCenterId - 1, HostOrPoolId - 1,
> PodId 1
> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing storage
> capacity update
> 2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity updates
> for public ip and Vlans
> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done capacity updates for
> public ip and Vlans
> 2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity updates
> for private ip
> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing capacity
> updates for private ip
> 2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done recalculating system
> capacity
> 2019-08-28 14:35:44,982 DEBUG [c.c.a.AlertManagerImpl]
> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Done running Capacity
> Checker ...
> 2019-08-28 14:35:49,922 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7f369e38) (logid:88e96986) Resetting hosts suitable for reconnect
> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7f369e38) (logid:88e96986) Completed resetting hosts suitable for
> reconnect
> 2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7f369e38) (logid:88e96986) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7f369e38) (logid:88e96986) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7f369e38) (logid:88e96986) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-7f369e38) (logid:88e96986) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-28 14:35:49,953 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-5e476ef2) (logid:be235fb1) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
> 2019-08-28 14:35:49,961 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:50,014 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
> 2019-08-28 14:35:50,633 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-3f36683f) (logid:7cc15823) ===START===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
> 2019-08-28 14:35:50,643 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 14:35:50,698 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) ===END===
> 137.74.35.102 -- GET
> command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
> 2019-08-28 14:35:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) Begin cleanup
> expired async-jobs
> 2019-08-28 14:35:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) End cleanup expired
> async-jobs
> 2019-08-28 14:36:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) Begin cleanup
> expired async-jobs
> 2019-08-28 14:36:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) End cleanup expired
> async-jobs
> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-318) has been pending for
> 1079 seconds
> 2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-320) has been pending for
> 477 seconds
> 2019-08-28 14:36:14,855 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) Begin cleanup
> expired async-jobs
> 2019-08-28 14:36:14,855 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 routers to
> update status.
> 2019-08-28 14:36:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 14:36:14,859 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 networks to
> update RvR status.
> 2019-08-28 14:36:14,864 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) End cleanup expired
> async-jobs
> 2019-08-28 14:36:24,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) Begin cleanup
> expired async-jobs
> 2019-08-28 14:36:24,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) End cleanup expired
> async-jobs
> 2019-08-28 14:36:24,972 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> 2019-08-28 14:36:24,972 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) Process host VM state report from
> ping process. host: 1
> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) Process VM state report. host: 1,
> number of records in report: 1
> 2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) VM state report. host: 1, vm id:
> 103, power state: PowerOn
> 2019-08-28 14:36:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
> vm id: 103, power state: PowerOn
> 2019-08-28 14:36:24,976 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 103, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:36:24,978 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) Run missing VM report. current time:
> 1566995784978
> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm id:
> 101, power state: PowerReportMissing, last state update: 1566995664000
> 2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) vm id: 101 - time since last state
> update(120978ms) has passed graceful period
> 2019-08-28 14:36:24,981 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
> vm id: 101, power state: PowerReportMissing
> 2019-08-28 14:36:24,981 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 101, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm id:
> 102, power state: PowerReportMissing, last state update: 1566995664000
> 2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) vm id: 102 - time since last state
> update(120978ms) has passed graceful period
> 2019-08-28 14:36:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
> vm id: 102, power state: PowerReportMissing
> 2019-08-28 14:36:24,984 INFO [c.c.v.VirtualMachineManagerImpl]
> (AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
> working on the VM. vm id: 102, postpone power-change report by resetting
> power-change counters
> 2019-08-28 14:36:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-7:null) (logid:) Done with process of VM state
> report. host: 1
> 2019-08-28 14:36:30,181 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-9c8e3bed) (logid:9058cad8) AutoScaling Monitor is
> running...
> 2019-08-28 14:36:34,848 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) Begin cleanup
> expired async-jobs
> 2019-08-28 14:36:34,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) End cleanup expired
> async-jobs
> 2019-08-28 14:36:37,458 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) HostStatsCollector is
> running...
> 2019-08-28 14:36:37,515 DEBUG [c.c.a.t.Request]
> (StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) Seq 1-5778962746846479046:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2019-08-28 14:36:38,661 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) StorageCollector is
> running...
> 2019-08-28 14:36:38,671 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) There is no secondary
> storage VM for secondary storage host Secondary1
> 2019-08-28 14:36:38,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) getCommandHostDelegation:
> class com.cloud.agent.api.GetStorageStatsCommand
> 2019-08-28 14:36:38,675 DEBUG [c.c.h.XenServerGuru]
> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) We are returning the
> default host to execute commands because the command is not of Copy type.
> 2019-08-28 14:36:38,729 DEBUG [c.c.a.t.Request]
> (StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) Seq 1-5778962746846479047:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>
> On Wed, Aug 28, 2019 at 4:04 PM Paul Angus <pa...@shapeblue.com>
> wrote:
>
>> The error is here Fariborz,
>>
>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
>>
>> you're already using a VLAN that you've also asked CloudStack to use.
>>
>>
>>
>> paul.angus@shapeblue.com
>> www.shapeblue.com
>> Amadeus House, Floral Street, London WC2E 9DPUK
>> @shapeblue
>>
>>
>>
>>
>> -----Original Message-----
>> From: Fariborz Navidan <md...@gmail.com>
>> Sent: 28 August 2019 12:29
>> To: users@cloudstack.apache.org
>> Subject: Unable to start System VMs
>>
>> Hello,
>>
>> New ACS management is unable to start System VMs. Below are logs. ACS is
>> still poor in error handling and reporting. It should be able to retyrn
>> clear error messages inside UI.
>>
>> 2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
>> 2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) ===END===
>> 137.74.35.102 -- GET
>> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
>> 2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START===
>> 137.74.35.102
>> -- GET
>>
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
>> 2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) ===END===
>> 137.74.35.102 -- GET
>>
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
>> 2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
>> 137.74.35.102 -- GET
>>
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
>> 2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) ===END===
>> 137.74.35.102 -- GET
>>
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
>> 2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START===
>> 137.74.35.102
>> -- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128967
>> 2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api
>> Discovery
>> plugin was unable to find an api by that name or process any apis
>> 2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) ===END===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
>> 2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:22:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers to
>> update status.
>> 2019-08-28 13:22:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 13:22:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 networks to
>> update RvR status.
>> 2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
>> 137.74.35.102 -- GET
>>
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
>> 2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) ===END===
>> 137.74.35.102 -- GET
>>
>> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
>> 2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START===
>> 137.74.35.102
>> -- GET
>>
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
>> 2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) ===END===
>> 137.74.35.102 -- GET
>>
>> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
>> 2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
>> 137.74.35.102 -- GET
>>
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
>> 2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) ===END===
>> 137.74.35.102 -- GET
>>
>> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
>> 2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START===
>> 137.74.35.102
>> -- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128971
>> 2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api
>> Discovery
>> plugin was unable to find an api by that name or process any apis
>> 2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) ===END===
>> 137.74.35.102 -- GET
>> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
>> 2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for
>> reconnect
>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts suitable
>> for
>> reconnect
>> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters already
>> owned by this management server
>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
>> clusters
>> already owned by this management server
>> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not
>> owned
>> by any management server
>> 2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for
>> clusters
>> not owned by any management server
>> 2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128972
>> 2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128972
>> 2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-3:null) (logid:) Process host VM state report from
>> ping process. host: 1
>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
>> number of records in report: 0
>> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor is
>> running...
>> 2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START===
>> 137.74.35.102
>> -- GET command=listInfrastructure&response=json&_=1566988128973
>> 2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128973
>> 2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector is
>> running...
>> 2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
>> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq
>> 1-5778962746846478713:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
>> running...
>> 2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no secondary
>> storage VM for secondary storage host Secondary1
>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) getCommandHostDelegation:
>> class com.cloud.agent.api.GetStorageStatsCommand
>> 2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
>> default host to execute commands because the command is not of Copy type.
>> 2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
>> (StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq
>> 1-5778962746846478714:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>> 2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to launch
>> secondary storage VM
>> 2019-08-28 13:22:40,562 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary storage
>> vms found in datacenter id=1, starting one
>> 2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary storage
>> vm is available, need to allocate a new secondary storage vm
>> 2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage vm
>> from a newly started instance for request from data center : 1
>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>> configuration for offering [Network Offering
>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
>> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>> configuration for offering [Network Offering
>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
>> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
>> configuration for offering [Network Offering
>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
>> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to launch
>> console proxy
>> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Expand console proxy
>> standby
>> capacity for zone France
>> 2019-08-28 13:22:40,615 INFO [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) No stopped console proxy is
>> available, need to allocate a new console proxy
>> 2019-08-28 13:22:40,616 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Assign console proxy from a
>> newly started instance for request from data center : 1
>> 2019-08-28 13:22:40,623 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating entries for VM:
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:22:40,625 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nics for
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:22:40,628 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
>> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[203|Guest|6] with requested
>> profile NicProfile[0-0-null-null-null
>> 2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
>> configuration for offering [Network Offering
>> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
>> 2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> 2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
>> configuration for offering [Network Offering
>> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
>> 2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> 2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
>> configuration for offering [Network Offering
>> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
>> 2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
>> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
>> 2019-08-28 13:22:40,660 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
>> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[201|Control|3] with
>> requested profile null
>> 2019-08-28 13:22:40,668 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
>> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[200|Management|2] with
>> requested profile null
>> 2019-08-28 13:22:40,673 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating entries for VM:
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:22:40,675 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nics for
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:22:40,676 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
>> VM[ConsoleProxy|v-68-VM] in network Ntwk[203|Guest|6] with requested
>> profile NicProfile[0-0-null-null-null
>> 2019-08-28 13:22:40,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating disks for
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:22:40,681 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocation completed for VM:
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
>> alert
>> 2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) New secondary storage vm
>> created, zone: France, secStorageVm: s-67-VM, public IP: null, private IP:
>> null
>> 2019-08-28 13:22:40,688 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
>> VM[ConsoleProxy|v-68-VM] in network Ntwk[201|Control|3] with requested
>> profile null
>> 2019-08-28 13:22:40,690 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Sync job-218 execution on
>> object VmWorkJobQueue.67
>> 2019-08-28 13:22:40,691 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
>> VM[ConsoleProxy|v-68-VM] in network Ntwk[200|Management|2] with requested
>> profile null
>> 2019-08-28 13:22:40,694 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating disks for
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:22:40,696 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocation completed for
>> VM:
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy
>> alert
>> 2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) New console proxy created,
>> zone: France, proxy: v-68-VM, public IP: null, private IP: null
>> 2019-08-28 13:22:40,701 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Sync job-219 execution on
>> object VmWorkJobQueue.68
>> 2019-08-28 13:22:40,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
>> item: SyncQueueItemVO {id:194, queueId: 191, contentType: AsyncJob,
>> contentId: 218, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
>> lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
>> 13:22:40 CEST 2019}
>> 2019-08-28 13:22:40,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
>> job-218
>> 2019-08-28 13:22:40,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
>> item: SyncQueueItemVO {id:195, queueId: 192, contentType: AsyncJob,
>> contentId: 219, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
>> lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
>> 13:22:40 CEST 2019}
>> 2019-08-28 13:22:40,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
>> job-219
>> 2019-08-28 13:22:40,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:d5030cfc) Add
>> job-218 into job monitoring
>> 2019-08-28 13:22:40,872 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:8371f7e8) Add
>> job-219 into job monitoring
>> 2019-08-28 13:22:40,877 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>> Executing AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType: null,
>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>> null}
>> 2019-08-28 13:22:40,877 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Run
>> VM
>> work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
>> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Execute VM work job:
>>
>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:22:40,878 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>> Executing AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType: null,
>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>> null}
>> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Run
>> VM
>> work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
>> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Execute VM work job:
>>
>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:22:40,880 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) VM state transitted from :Stopped to Starting with event:
>> StartRequestedvm's original host id: null new host id: null host id before
>> state transition: null
>> 2019-08-28 13:22:40,880 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Successfully transitioned to start state for
>> VM[SecondaryStorageVm|s-67-VM] reservation id =
>> c608b88c-7d06-411e-bb84-f2d5e2b3390d
>> 2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:22:40,881 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
>> StartRequestedvm's original host id: null new host id: null host id before
>> state transition: null
>> 2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Successfully transitioned to start state for
>> VM[ConsoleProxy|v-68-VM] reservation id =
>> 6c836a9c-0d16-42c3-86c7-446febb1a189
>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>> pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
>> 2019-08-28 13:22:40,882 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Searching all possible resources under this Zone: 1
>> 2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Listing clusters in order of aggregate capacity, that
>> have
>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>> pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Searching all possible resources under this Zone: 1
>> 2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that
>> have
>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>> 2019-08-28 13:22:40,886 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:22:40,886 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:22:40,887 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>> Ram=512
>> 2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>> Ram=1024
>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
>> capacity for requested CPU: 500 and requested RAM: 536870912 ,
>> cpuOverprovisioningFactor: 4.0
>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
>> and CPU after applying overprovisioning: 216000
>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
>> CPU: 500
>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
>> Requested RAM: 536870912
>> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
>> available
>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
>> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
>> 216000; requested cpu:500,alloc_from_last_host?:false
>> ,considerReservedCapacity?: true
>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>> 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
>> 2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding
>> to
>> list: 1
>> 2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
>> suitable hosts
>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) We need to allocate new storagepool for this volume
>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Calling StoragePoolAllocators to find suitable pools
>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
>> capacity for requested CPU: 500 and requested RAM: 1073741824 ,
>> cpuOverprovisioningFactor: 4.0
>> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) System VMs will use shared storage for zone id=1
>> 2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) LocalStoragePoolAllocator trying to find storage pool to
>> fit the vm
>> 2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) ClusterScopeStoragePoolAllocator looking for storage pool
>> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
>> pools will be ignored.
>> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Found pools matching tags: [Pool[1|SharedMountPoint]]
>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
>> and CPU after applying overprovisioning: 216000
>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
>> CPU: 500
>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
>> Requested RAM: 1073741824
>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
>> available
>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
>> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
>> 216000; requested cpu:500,alloc_from_last_host?:false
>> ,considerReservedCapacity?: true
>> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>> 1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
>> 2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding
>> to
>> list: 1
>> 2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
>> suitable hosts
>> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking if storage pool is suitable, name: null
>> ,poolId: 1
>> 2019-08-28 13:22:40,891 INFO [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Storage pool null (1) does not supply IOPS capacity,
>> assuming enough capacity
>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) We need to allocate new storagepool for this volume
>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Calling StoragePoolAllocators to find suitable pools
>> 2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking pool 1 for storage, totalSize: 984895062016,
>> usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold:
>> 0.85
>> 2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Destination pool id: 1
>> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) System VMs will use shared storage for zone id=1
>> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) LocalStoragePoolAllocator trying to find storage pool to
>> fit the vm
>> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) ClusterScopeStoragePoolAllocator looking for storage pool
>> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
>> pools will be ignored.
>> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Found pools matching tags: [Pool[1|SharedMountPoint]]
>> 2019-08-28 13:22:40,893 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking if storage pool is suitable, name: null
>> ,poolId: 1
>> 2019-08-28 13:22:40,893 INFO [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Storage pool null (1) does not supply IOPS capacity,
>> assuming enough capacity
>> 2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking pool 1 for storage, totalSize: 984895062016,
>> usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold:
>> 0.85
>> 2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Destination pool id: 1
>> 2019-08-28 13:22:40,896 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Pool ID for the volume with ID 62 is null
>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Found storage pool Primary1 of type SharedMountPoint
>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Total capacity of the pool Primary1 with ID 1 is
>> 984895062016
>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking pool: 1 for storage allocation , maxSize :
>> 984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
>> disable threshold: 0.85
>> 2019-08-28 13:22:40,897 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) ClusterScopeStoragePoolAllocator returning 1 suitable
>> storage pools
>> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Trying to find a potenial host and associated storage
>> pools from the suitable host/pool lists for this VM
>> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking if host: 1 can access any suitable storage pool
>> for volume: ROOT
>> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Host: 1 can access pool: 1
>> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Pool ID for the volume with ID 63 is null
>> 2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Found storage pool Primary1 of type SharedMountPoint
>> 2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Total capacity of the pool Primary1 with ID 1 is
>> 984895062016
>> 2019-08-28 13:22:40,898 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com
>> and
>> associated storage pools for this VM
>> 2019-08-28 13:22:40,899 DEBUG [c.c.s.StorageManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking pool: 1 for storage allocation , maxSize :
>> 984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
>> disable threshold: 0.85
>> 2019-08-28 13:22:40,899 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) ClusterScopeStoragePoolAllocator returning 1 suitable
>> storage pools
>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Trying to find a potenial host and associated storage
>> pools from the suitable host/pool lists for this VM
>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
>> for volume: ROOT
>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Returning Deployment Destination:
>>
>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> :
>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
>> 2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
>>
>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> :
>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Host: 1 can access pool: 1
>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com
>> and
>> associated storage pools for this VM
>> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Returning Deployment Destination:
>>
>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> :
>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
>> 2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
>>
>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> :
>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
>> 2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) VM state transitted from :Starting to Starting with
>> event:
>> OperationRetryvm's original host id: null new host id: 1 host id before
>> state transition: null
>> 2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) VM state transitted from :Starting to Starting with
>> event:
>> OperationRetryvm's original host id: null new host id: 1 host id before
>> state transition: null
>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) We are allocating VM, increasing the used capacity of
>> this
>> host:1
>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested
>> RAM:
>> 536870912
>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
>> reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
>> used:500, reserved:0; requested cpu:500,alloc_from_last:false
>> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
>> reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
>> requested mem: 536870912,alloc_from_last:false
>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) We are allocating VM, increasing the used capacity of
>> this
>> host:1
>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU:
>> 500
>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
>> ,Requested RAM: 1073741824
>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
>> old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
>> new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
>> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
>> 536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
>> reserved: 0; requested mem: 1073741824,alloc_from_last:false
>> 2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Network id=203 is already implemented
>> 2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Network id=203 is already implemented
>> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Asking VirtualRouter to prepare for
>> Nic[184-67-null-137.74.82.208]
>> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Asking VirtualRouter to prepare for
>> Nic[187-68-null-137.74.82.209]
>> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Asking SecurityGroupProvider to prepare for
>> Nic[184-67-null-137.74.82.208]
>> 2019-08-28 13:22:40,913 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Asking SecurityGroupProvider to prepare for
>> Nic[187-68-null-137.74.82.209]
>> 2019-08-28 13:22:40,914 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Network id=201 is already implemented
>> 2019-08-28 13:22:40,915 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Network id=201 is already implemented
>> 2019-08-28 13:22:40,919 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Network id=200 is already implemented
>> 2019-08-28 13:22:40,920 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Network id=200 is already implemented
>> 2019-08-28 13:22:40,922 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Releasing ip address for instance=186
>> 2019-08-28 13:22:40,923 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Releasing ip address for instance=189
>> 2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Allocated a nic
>> NicProfile[186-67-c608b88c-7d06-411e-bb84-f2d5e2b3390d-192.168.0.226-null
>> for VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Allocated a nic
>> NicProfile[189-68-6c836a9c-0d16-42c3-86c7-446febb1a189-192.168.0.121-null
>> for VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) template 3 is already in store:1, type:Image
>> 2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) template 3 is already in store:1, type:Image
>> 2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) template 3 is already in store:1, type:Primary
>> 2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) template 3 is already in store:1, type:Primary
>> 2019-08-28 13:22:40,932 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) copyAsync inspecting src type TEMPLATE copyAsync
>> inspecting dest type VOLUME
>> 2019-08-28 13:22:40,933 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) copyAsync inspecting src type TEMPLATE copyAsync
>> inspecting dest type VOLUME
>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) getCommandHostDelegation: class
>> org.apache.cloudstack.storage.command.CopyCommand
>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) We are returning the default host to execute commands
>> because the target hypervisor of the source data is not XenServer.
>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) getCommandHostDelegation: class
>> org.apache.cloudstack.storage.command.CopyCommand
>> 2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) We are returning the default host to execute commands
>> because the target hypervisor of the source data is not XenServer.
>> 2019-08-28 13:22:40,934 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Seq 1-5778962746846478715: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
>>
>> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
>> cloudstack.storage.to
>> .TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
>>
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
>> ","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
>> Template
>> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
>> }
>> 2019-08-28 13:22:40,935 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Seq 1-5778962746846478716: Waiting for Seq
>> 5778962746846478715 Scheduling: { Cmd , MgmtId: 205425461278937, via: 1(
>> fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
>>
>> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
>> cloudstack.storage.to
>> .TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
>>
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
>> ","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
>> Template
>> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
>> }
>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-138d8f65) (logid:85352d35) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128974
>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478715:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>> 110,
>>
>> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
>> }
>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.m.AgentAttache]
>> (AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478716: Sending
>> now. is current sequence.
>> 2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Seq 1-5778962746846478715: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
>> CopyCmdAnswer } }
>> 2019-08-28 13:22:41,040 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:41,047 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128974
>> 2019-08-28 13:22:41,057 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
>> template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
>> zone=1 pod=1 guid=s-67-VM workers=5
>>
>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>> dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
>> 2019-08-28 13:22:41,071 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Seq 1-5778962746846478717: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>
>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>> dns1=8.8.8.8 dns2=8.8.4.4
>>
>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
>> }
>> 2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags:
>> 110,
>>
>> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
>> }
>> 2019-08-28 13:22:41,126 DEBUG [c.c.a.m.AgentAttache]
>> (AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716: No more
>> commands found
>> 2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Seq 1-5778962746846478716: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
>> CopyCmdAnswer } }
>> 2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-9:null) (logid:) Seq 1-5778962746846478717:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>>
>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>
>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>> dns1=8.8.8.8 dns2=8.8.4.4
>>
>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>> to create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>
>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>> by previous failure","wait":0}}] }
>> 2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Seq 1-5778962746846478717: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StartAnswer, Answer } }
>> 2019-08-28 13:22:41,128 INFO [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
>> 2019-08-28 13:22:41,129 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Cleaning up resources for the vm
>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>> 2019-08-28 13:22:41,131 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Seq 1-5778962746846478718: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.64","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
>> }
>> 2019-08-28 13:22:41,139 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
>> type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
>> guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
>> eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.2.191
>> eth0mask=255.255.0.0 eth1ip=192.168.0.121 eth1mask=255.255.255.0 mgmtcidr=
>> 51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
>> internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
>> 2019-08-28 13:22:41,151 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Seq 1-5778962746846478719: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>> port=8250
>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>
>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.191","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
>> }
>> 2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-8:null) (logid:) Seq 1-5778962746846478719:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>>
>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>> port=8250
>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>
>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>> to create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>
>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>> by previous failure","wait":0}}] }
>> 2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Seq 1-5778962746846478719: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StartAnswer, Answer } }
>> 2019-08-28 13:22:41,323 INFO [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
>> 2019-08-28 13:22:41,326 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
>> in Starting state
>> 2019-08-28 13:22:41,337 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Seq 1-5778962746846478720: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.2.191","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
>> }
>> 2019-08-28 13:22:44,313 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-10:null) (logid:) Seq 1-5778962746846478718:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>> 2019-08-28 13:22:44,314 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Seq 1-5778962746846478718: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StopAnswer } }
>> 2019-08-28 13:22:44,318 DEBUG [c.c.n.g.ControlNetworkGuru]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
>> 2019-08-28 13:22:44,321 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Releasing ip address for
>> reservationId=c608b88c-7d06-411e-bb84-f2d5e2b3390d, instance=186
>> 2019-08-28 13:22:44,321 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
>> 2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Successfully released network resources for the vm
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Successfully cleaned up resources for the VM
>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>> 2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Root volume is ready, need to place VM in volume's
>> cluster
>> 2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
>> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
>> 2019-08-28 13:22:44,324 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
>> 2019-08-28 13:22:44,324 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:22:44,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:22:44,327 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:22:44,328 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>> Ram=512
>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
>> suitable hosts
>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) No suitable hosts found
>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) No suitable hosts found under this Cluster: 1
>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Could not find suitable Deployment Destination for this
>> VM
>> under any clusters, returning.
>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) The specified cluster is in avoid set, returning.
>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>> pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
>> 2019-08-28 13:22:44,331 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Searching all possible resources under this Zone: 1
>> 2019-08-28 13:22:44,332 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Listing clusters in order of aggregate capacity, that
>> have
>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>> 2019-08-28 13:22:44,333 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Removing from the clusterId list these clusters from
>> avoid
>> set: [1]
>> 2019-08-28 13:22:44,334 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) No clusters found after removing disabled clusters and
>> clusters in avoid list, returning.
>> 2019-08-28 13:22:44,338 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) VM state transitted from :Starting to Stopped with event:
>> OperationFailedvm's original host id: null new host id: null host id
>> before
>> state transition: 1
>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
>> applying overprovisioning: 66088833024
>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
>> actual total: 54000, total with overprovisioning: 216000; new used:
>> 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
>> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
>> 0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
>> false,moveToReserveredfalse
>> 2019-08-28 13:22:44,343 ERROR [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Invocation exception, caused by:
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:22:44,343 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
>> (logid:e3068c13) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:22:44,343 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin:
>> 76
>> 2019-08-28 13:22:44,344 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>> Unable
>> to complete AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType:
>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>> null}, job origin:76
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:22:44,344 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>> Complete async job-218, jobStatus: FAILED, resultCode: 0, result:
>>
>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>> Publish async job-218 complete on message bus
>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
>> up jobs related to job-218
>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>> Update
>> db status for job-218
>> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
>> up jobs joined with job-218 and disjoin all subjobs created from job- 218
>> 2019-08-28 13:22:44,349 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
>> executing com.cloud.vm.VmWorkStart for job-218
>> 2019-08-28 13:22:44,350 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
>> Remove
>> job-218 from job monitoring
>> 2019-08-28 13:22:44,351 WARN [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Exception while trying to
>> start secondary storage vm
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:22:44,352 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Unable to start secondary
>> storage vm for standby capacity, vm id : 67, will recycle it and start a
>> new one
>> 2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
>> alert
>> 2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Secondary Storage Vm creation
>> failure, zone: France
>> 2019-08-28 13:22:44,352 WARN [c.c.a.AlertManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) AlertType:: 19 |
>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
>> Storage Vm creation failure. zone: France, error details: null
>> 2019-08-28 13:22:44,353 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Primary secondary storage is
>> not even started, wait until next turn
>> 2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-11:null) (logid:) Seq 1-5778962746846478720:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>> 2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Seq 1-5778962746846478720: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StopAnswer } }
>> 2019-08-28 13:22:44,436 DEBUG [c.c.n.g.ControlNetworkGuru]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
>> 2019-08-28 13:22:44,438 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Releasing ip address for
>> reservationId=6c836a9c-0d16-42c3-86c7-446febb1a189, instance=189
>> 2019-08-28 13:22:44,438 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
>> 2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Successfully released network resources for the vm
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Successfully cleaned up resources for the VM
>> VM[ConsoleProxy|v-68-VM] in Starting state
>> 2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Root volume is ready, need to place VM in volume's
>> cluster
>> 2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
>> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
>> 2019-08-28 13:22:44,441 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:22:44,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:22:44,443 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>> Ram=1024
>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
>> suitable hosts
>> 2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) No suitable hosts found
>> 2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) No suitable hosts found under this Cluster: 1
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Could not find suitable Deployment Destination for this
>> VM
>> under any clusters, returning.
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) The specified cluster is in avoid set, returning.
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>> pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
>> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
>> 2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Searching all possible resources under this Zone: 1
>> 2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that
>> have
>> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
>> 2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Removing from the clusterId list these clusters from
>> avoid
>> set: [1]
>> 2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) No clusters found after removing disabled clusters and
>> clusters in avoid list, returning.
>> 2019-08-28 13:22:44,450 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
>> OperationFailedvm's original host id: null new host id: null host id
>> before
>> state transition: 1
>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
>> applying overprovisioning: 66088833024
>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
>> actual total: 54000, total with overprovisioning: 216000; new used:
>> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
>> 0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
>> false,moveToReserveredfalse
>> 2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Invocation exception, caused by:
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:22:44,453 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
>> (logid:3e502c4b) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:22:44,453 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin:
>> 77
>> 2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>> Unable
>> to complete AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType:
>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
>> null}, job origin:77
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>> Complete async job-219, jobStatus: FAILED, resultCode: 0, result:
>>
>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>> Publish async job-219 complete on message bus
>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
>> up jobs related to job-219
>> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>> Update
>> db status for job-219
>> 2019-08-28 13:22:44,455 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
>> up jobs joined with job-219 and disjoin all subjobs created from job- 219
>> 2019-08-28 13:22:44,457 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
>> executing com.cloud.vm.VmWorkStart for job-219
>> 2019-08-28 13:22:44,458 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
>> Remove
>> job-219 from job monitoring
>> 2019-08-28 13:22:44,459 WARN [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Exception while trying to
>> start console proxy
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:22:44,460 INFO [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Unable to start console
>> proxy vm for standby capacity, vm id : 68, will recycle it and start a new
>> one
>> 2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy
>> alert
>> 2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Console proxy creation
>> failure, zone: France
>> 2019-08-28 13:22:44,460 WARN [c.c.a.AlertManagerImpl]
>> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) AlertType:: 10 |
>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
>> proxy creation failure. zone: France, error details: null
>> 2019-08-28 13:22:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:22:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 routers to
>> update status.
>> 2019-08-28 13:22:44,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 13:22:44,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 networks to
>> update RvR status.
>> 2019-08-28 13:22:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:48,876 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-1ca6b8a4) (logid:767cda38) ===START===
>> 137.74.35.102
>> -- GET command=listInfrastructure&response=json&_=1566988128975
>> 2019-08-28 13:22:48,883 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:48,903 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128975
>> 2019-08-28 13:22:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:22:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:22:56,741 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-3fa512ab) (logid:4609e172) ===START===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128976
>> 2019-08-28 13:22:56,747 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:22:56,751 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128976
>> 2019-08-28 13:23:03,063 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-a0c69e67) (logid:0a2b6c35) ===START===
>> 137.74.35.102
>> -- GET command=listInfrastructure&response=json&_=1566988128977
>> 2019-08-28 13:23:03,071 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:23:03,091 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128977
>> 2019-08-28 13:23:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:23:04,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:23:09,900 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-16:ctx-9f3738f2) (logid:dea95b1b) ===START===
>> 137.74.35.102
>> -- GET command=listInfrastructure&response=json&_=1566988128978
>> 2019-08-28 13:23:09,907 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:23:09,929 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128978
>> 2019-08-28 13:23:10,557 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Zone 1 is ready to launch
>> secondary storage VM
>> 2019-08-28 13:23:10,561 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) No running secondary storage
>> vms found in datacenter id=1, starting one
>> 2019-08-28 13:23:10,570 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Found a stopped secondary
>> storage vm, starting it. Vm id : 67
>> 2019-08-28 13:23:10,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-220 execution on
>> object VmWorkJobQueue.67
>> 2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Zone 1 is ready to launch
>> console proxy
>> 2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expand console proxy
>> standby
>> capacity for zone France
>> 2019-08-28 13:23:10,616 INFO [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Found a stopped console
>> proxy, starting it. Vm id : 68
>> 2019-08-28 13:23:10,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-221 execution on
>> object VmWorkJobQueue.68
>> 2019-08-28 13:23:10,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
>> item: SyncQueueItemVO {id:196, queueId: 191, contentType: AsyncJob,
>> contentId: 220, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
>> lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
>> 13:23:10 CEST 2019}
>> 2019-08-28 13:23:10,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
>> job-220
>> 2019-08-28 13:23:10,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
>> item: SyncQueueItemVO {id:197, queueId: 192, contentType: AsyncJob,
>> contentId: 221, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
>> lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
>> 13:23:10 CEST 2019}
>> 2019-08-28 13:23:10,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
>> job-221
>> 2019-08-28 13:23:10,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:f78794fa) Add
>> job-220 into job monitoring
>> 2019-08-28 13:23:10,871 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:7deee6cf) Add
>> job-221 into job monitoring
>> 2019-08-28 13:23:10,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>> Executing AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType: null,
>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>> null}
>> 2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Run
>> VM
>> work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
>> 2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Execute VM work job:
>>
>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:23:10,881 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>> Executing AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType: null,
>> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>> null}
>> 2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Run
>> VM
>> work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
>> 2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Execute VM work job:
>>
>> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:23:10,883 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) VM state transitted from :Stopped to Starting with event:
>> StartRequestedvm's original host id: null new host id: null host id before
>> state transition: null
>> 2019-08-28 13:23:10,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Successfully transitioned to start state for
>> VM[SecondaryStorageVm|s-67-VM] reservation id =
>> 46a0e954-a9af-45af-ac39-91e036654f95
>> 2019-08-28 13:23:10,884 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
>> StartRequestedvm's original host id: null new host id: null host id before
>> state transition: null
>> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Successfully transitioned to start state for
>> VM[ConsoleProxy|v-68-VM] reservation id =
>> a25c0118-b49c-41b2-8970-9b73191746f3
>> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Root volume is ready, need to place VM in volume's
>> cluster
>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Root volume is ready, need to place VM in volume's
>> cluster
>> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
>> 2019-08-28 13:23:10,886 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>> Ram=1024
>> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>> Ram=512
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
>> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
>> capacity for requested CPU: 500 and requested RAM: 1073741824 ,
>> cpuOverprovisioningFactor: 4.0
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
>> capacity for requested CPU: 500 and requested RAM: 536870912 ,
>> cpuOverprovisioningFactor: 4.0
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
>> and CPU after applying overprovisioning: 216000
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
>> CPU: 500
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
>> Requested RAM: 1073741824
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
>> available
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
>> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
>> 216000; requested cpu:500,alloc_from_last_host?:false
>> ,considerReservedCapacity?: true
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>> 1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding
>> to
>> list: 1
>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
>> suitable hosts
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
>> and CPU after applying overprovisioning: 216000
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
>> CPU: 500
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
>> Requested RAM: 536870912
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
>> available
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
>> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
>> 216000; requested cpu:500,alloc_from_last_host?:false
>> ,considerReservedCapacity?: true
>> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
>> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
>> 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding
>> to
>> list: 1
>> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
>> suitable hosts
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Volume has pool already allocated, checking if pool can
>> be
>> reused, poolId: 1
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Volume has pool already allocated, checking if pool can
>> be
>> reused, poolId: 1
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Planner need not allocate a pool for this volume since
>> its
>> READY
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Trying to find a potenial host and associated storage
>> pools from the suitable host/pool lists for this VM
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
>> for volume: ROOT
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Planner need not allocate a pool for this volume since
>> its
>> READY
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Trying to find a potenial host and associated storage
>> pools from the suitable host/pool lists for this VM
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Checking if host: 1 can access any suitable storage pool
>> for volume: ROOT
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Host: 1 can access pool: 1
>> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Host: 1 can access pool: 1
>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com
>> and
>> associated storage pools for this VM
>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com
>> and
>> associated storage pools for this VM
>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Returning Deployment Destination:
>>
>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>> 2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
>>
>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Returning Deployment Destination:
>>
>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>> 2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
>>
>> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
>> 2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) VM state transitted from :Starting to Starting with
>> event:
>> OperationRetryvm's original host id: null new host id: 1 host id before
>> state transition: null
>> 2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) VM state transitted from :Starting to Starting with
>> event:
>> OperationRetryvm's original host id: null new host id: 1 host id before
>> state transition: null
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) We are allocating VM, increasing the used capacity of
>> this
>> host:1
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested
>> RAM:
>> 536870912
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
>> reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
>> used:500, reserved:0; requested cpu:500,alloc_from_last:false
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
>> reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
>> requested mem: 536870912,alloc_from_last:false
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) We are allocating VM, increasing the used capacity of
>> this
>> host:1
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU:
>> 500
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
>> ,Requested RAM: 1073741824
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
>> old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
>> new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
>> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
>> 536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
>> reserved: 0; requested mem: 1073741824,alloc_from_last:false
>> 2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Network id=203 is already implemented
>> 2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Network id=203 is already implemented
>> 2019-08-28 13:23:10,907 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Asking VirtualRouter to prepare for
>> Nic[187-68-null-137.74.82.209]
>> 2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Asking VirtualRouter to prepare for
>> Nic[184-67-null-137.74.82.208]
>> 2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Asking SecurityGroupProvider to prepare for
>> Nic[187-68-null-137.74.82.209]
>> 2019-08-28 13:23:10,909 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Asking SecurityGroupProvider to prepare for
>> Nic[184-67-null-137.74.82.208]
>> 2019-08-28 13:23:10,911 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Network id=201 is already implemented
>> 2019-08-28 13:23:10,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Network id=201 is already implemented
>> 2019-08-28 13:23:10,916 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Network id=200 is already implemented
>> 2019-08-28 13:23:10,918 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Network id=200 is already implemented
>> 2019-08-28 13:23:10,918 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Releasing ip address for instance=186
>> 2019-08-28 13:23:10,920 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Allocated a nic
>> NicProfile[186-67-46a0e954-a9af-45af-ac39-91e036654f95-192.168.0.86-null
>> for VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:23:10,921 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Releasing ip address for instance=189
>> 2019-08-28 13:23:10,922 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) No need to recreate the volume: Vol[62|vm=67|ROOT], since
>> it already has a pool assigned: 1, adding disk to VM
>> 2019-08-28 13:23:10,923 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Allocated a nic
>> NicProfile[189-68-a25c0118-b49c-41b2-8970-9b73191746f3-192.168.0.20-null
>> for VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:23:10,925 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) No need to recreate the volume: Vol[63|vm=68|ROOT], since
>> it already has a pool assigned: 1, adding disk to VM
>> 2019-08-28 13:23:10,930 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
>> template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
>> zone=1 pod=1 guid=s-67-VM workers=5
>>
>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>> dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
>> 2019-08-28 13:23:10,931 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
>> type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
>> guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
>> eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.0.248
>> eth0mask=255.255.0.0 eth1ip=192.168.0.20 eth1mask=255.255.255.0 mgmtcidr=
>> 51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
>> internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
>> 2019-08-28 13:23:10,941 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Seq 1-5778962746846478721: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>
>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>> dns1=8.8.8.8 dns2=8.8.4.4
>>
>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.170","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
>> }
>> 2019-08-28 13:23:10,942 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Seq 1-5778962746846478722: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>> port=8250
>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>
>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.248","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
>> }
>> 2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-12:null) (logid:) Seq 1-5778962746846478722:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>>
>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86
>> port=8250
>> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
>> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>>
>> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>> to create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>
>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>> by previous failure","wait":0}}] }
>> 2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Seq 1-5778962746846478722: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StartAnswer, Answer } }
>> 2019-08-28 13:23:10,985 INFO [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
>> 2019-08-28 13:23:10,986 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
>> in Starting state
>> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Seq 1-5778962746846478723: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.0.248","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
>> }
>> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-13:null) (logid:) Seq 1-5778962746846478721:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>>
>> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
>> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
>> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
>> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
>>
>> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
>> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
>> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
>> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
>> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
>> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
>> dns1=8.8.8.8 dns2=8.8.4.4
>>
>> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
>> to create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>>
>> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
>> by previous failure","wait":0}}] }
>> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Seq 1-5778962746846478721: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StartAnswer, Answer } }
>> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
>> create vnet 100: device eth1.100 is already a member of a bridge; can't
>> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
>> 2019-08-28 13:23:10,990 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Cleaning up resources for the vm
>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>> 2019-08-28 13:23:10,991 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Seq 1-5778962746846478724: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.170","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
>> }
>> 2019-08-28 13:23:14,130 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-14:null) (logid:) Seq 1-5778962746846478724:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>> 2019-08-28 13:23:14,131 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Seq 1-5778962746846478724: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StopAnswer } }
>> 2019-08-28 13:23:14,136 DEBUG [c.c.n.g.ControlNetworkGuru]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
>> 2019-08-28 13:23:14,138 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Releasing ip address for
>> reservationId=46a0e954-a9af-45af-ac39-91e036654f95, instance=186
>> 2019-08-28 13:23:14,139 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
>> 2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Successfully released network resources for the vm
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Successfully cleaned up resources for the VM
>> VM[SecondaryStorageVm|s-67-VM] in Starting state
>> 2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Root volume is ready, need to place VM in volume's
>> cluster
>> 2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
>> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
>> 2019-08-28 13:23:14,142 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:23:14,145 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:23:14,145 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:23:14,146 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
>> Ram=512
>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
>> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
>> suitable hosts
>> 2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) No suitable hosts found
>> 2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) No suitable hosts found under this Cluster: 1
>> 2019-08-28 13:23:14,148 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Could not find suitable Deployment Destination for this
>> VM
>> under any clusters, returning.
>> 2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) The specified cluster is in avoid set, returning.
>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:null, requested cpu: 500, requested ram: 536870912
>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Searching resources only under specified Pod: 1
>> 2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Listing clusters in order of aggregate capacity, that
>> have
>> (atleast one host with) enough CPU and RAM capacity under this Pod: 1
>> 2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Removing from the clusterId list these clusters from
>> avoid
>> set: [1]
>> 2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) No clusters found after removing disabled clusters and
>> clusters in avoid list, returning.
>> 2019-08-28 13:23:14,156 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) VM state transitted from :Starting to Stopped with event:
>> OperationFailedvm's original host id: null new host id: null host id
>> before
>> state transition: 1
>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
>> applying overprovisioning: 66088833024
>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
>> actual total: 54000, total with overprovisioning: 216000; new used:
>> 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
>> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
>> 0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
>> false,moveToReserveredfalse
>> 2019-08-28 13:23:14,160 ERROR [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Invocation exception, caused by:
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:23:14,161 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
>> (logid:e3068c13) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:23:14,161 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin:
>> 76
>> 2019-08-28 13:23:14,161 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>> Unable
>> to complete AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType:
>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>> null}, job origin:76
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:23:14,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>> Complete async job-220, jobStatus: FAILED, resultCode: 0, result:
>>
>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>> Publish async job-220 complete on message bus
>> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
>> up jobs related to job-220
>> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>> Update
>> db status for job-220
>> 2019-08-28 13:23:14,163 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
>> up jobs joined with job-220 and disjoin all subjobs created from job- 220
>> 2019-08-28 13:23:14,166 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
>> executing com.cloud.vm.VmWorkStart for job-220
>> 2019-08-28 13:23:14,166 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
>> Remove
>> job-220 from job monitoring
>> 2019-08-28 13:23:14,169 WARN [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Exception while trying to
>> start secondary storage vm
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:23:14,169 INFO [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Unable to start secondary
>> storage vm for standby capacity, vm id : 67, will recycle it and start a
>> new one
>> 2019-08-28 13:23:14,172 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-222 execution on
>> object VmWorkJobQueue.67
>> 2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-15:null) (logid:) Seq 1-5778962746846478723:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>> 2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Seq 1-5778962746846478723: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> StopAnswer } }
>> 2019-08-28 13:23:14,623 DEBUG [c.c.n.g.ControlNetworkGuru]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
>> 2019-08-28 13:23:14,626 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Releasing ip address for
>> reservationId=a25c0118-b49c-41b2-8970-9b73191746f3, instance=189
>> 2019-08-28 13:23:14,626 DEBUG [c.c.n.g.PodBasedNetworkGuru]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
>> 2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Successfully released network resources for the vm
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Successfully cleaned up resources for the VM
>> VM[ConsoleProxy|v-68-VM] in Starting state
>> 2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Root volume is ready, need to place VM in volume's
>> cluster
>> 2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
>> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
>> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
>> 2019-08-28 13:23:14,631 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:23:14,635 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
>> 2019-08-28 13:23:14,635 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
>> pod:1 cluster:1
>> 2019-08-28 13:23:14,636 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts
>> to
>> check for allocation: [Host[-1-Routing]]
>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
>> after prioritization: [Host[-1-Routing]]
>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
>> Ram=1024
>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
>> hostId: 1 is in avoid set, skipping this and trying other available hosts
>> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
>> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
>> suitable hosts
>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) No suitable hosts found
>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) No suitable hosts found under this Cluster: 1
>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Could not find suitable Deployment Destination for this
>> VM
>> under any clusters, returning.
>> 2019-08-28 13:23:14,638 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Searching resources only under specified Cluster: 1
>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) The specified cluster is in avoid set, returning.
>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
>> pod:1,cluster:null, requested cpu: 500, requested ram: 1073741824
>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
>> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
>> 2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Searching resources only under specified Pod: 1
>> 2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that
>> have
>> (atleast one host with) enough CPU and RAM capacity under this Pod: 1
>> 2019-08-28 13:23:14,641 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Removing from the clusterId list these clusters from
>> avoid
>> set: [1]
>> 2019-08-28 13:23:14,642 DEBUG [c.c.d.FirstFitPlanner]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) No clusters found after removing disabled clusters and
>> clusters in avoid list, returning.
>> 2019-08-28 13:23:14,646 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
>> OperationFailedvm's original host id: null new host id: null host id
>> before
>> state transition: 1
>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
>> overprovisioning: 216000
>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
>> applying overprovisioning: 66088833024
>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
>> actual total: 54000, total with overprovisioning: 216000; new used:
>> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
>> 0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
>> false,moveToReserveredfalse
>> 2019-08-28 13:23:14,650 ERROR [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Invocation exception, caused by:
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:23:14,651 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
>> (logid:3e502c4b) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2019-08-28 13:23:14,651 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin:
>> 77
>> 2019-08-28 13:23:14,651 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>> Unable
>> to complete AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType:
>> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>>
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
>> null}, job origin:77
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>> Complete async job-221, jobStatus: FAILED, resultCode: 0, result:
>>
>> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>> Publish async job-221 complete on message bus
>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
>> up jobs related to job-221
>> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>> Update
>> db status for job-221
>> 2019-08-28 13:23:14,653 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
>> up jobs joined with job-221 and disjoin all subjobs created from job- 221
>> 2019-08-28 13:23:14,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
>> executing com.cloud.vm.VmWorkStart for job-221
>> 2019-08-28 13:23:14,656 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
>> Remove
>> job-221 from job monitoring
>> 2019-08-28 13:23:14,658 WARN [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Exception while trying to
>> start console proxy
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a
>> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
>> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>> at
>>
>> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
>> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> 2019-08-28 13:23:14,658 INFO [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Unable to start console
>> proxy vm for standby capacity, vm id : 68, will recycle it and start a new
>> one
>> 2019-08-28 13:23:14,662 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-223 execution on
>> object VmWorkJobQueue.68
>> 2019-08-28 13:23:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 routers to
>> update status.
>> 2019-08-28 13:23:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 VPC networks
>> to update Redundant State.
>> 2019-08-28 13:23:14,855 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
>> item: SyncQueueItemVO {id:198, queueId: 191, contentType: AsyncJob,
>> contentId: 222, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
>> lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
>> 13:23:14 CEST 2019}
>> 2019-08-28 13:23:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 networks to
>> update RvR status.
>> 2019-08-28 13:23:14,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
>> job-222
>> 2019-08-28 13:23:14,859 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
>> item: SyncQueueItemVO {id:199, queueId: 192, contentType: AsyncJob,
>> contentId: 223, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
>> lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
>> 13:23:14 CEST 2019}
>> 2019-08-28 13:23:14,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
>> job-223
>> 2019-08-28 13:23:14,863 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:7e4a2784) Add
>> job-222 into job monitoring
>> 2019-08-28 13:23:14,869 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:26933ca4) Add
>> job-223 into job monitoring
>> 2019-08-28 13:23:14,871 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:23:14,875 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
>> Executing AsyncJobVO {id:222, userId: 1, accountId: 1, instanceType: null,
>> instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
>>
>> rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
>> null}
>> 2019-08-28 13:23:14,875 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Run
>> VM
>> work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
>> 2019-08-28 13:23:14,877 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Execute VM work job:
>>
>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:23:14,879 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:23:14,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
>> Executing AsyncJobVO {id:223, userId: 1, accountId: 1, instanceType: null,
>> instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
>>
>> rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
>> null}
>> 2019-08-28 13:23:14,881 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Run
>> VM
>> work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
>> 2019-08-28 13:23:14,882 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Execute VM work job:
>>
>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:23:14,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) VM is already stopped: VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:23:14,883 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Done executing VM work job:
>>
>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:23:14,883 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Complete async job-222, jobStatus: SUCCEEDED, resultCode:
>> 0, result: null
>> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Publish async job-222 complete on message bus
>> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Wake up jobs related to job-222
>> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Update db status for job-222
>> 2019-08-28 13:23:14,886 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
>> (logid:e3068c13) Wake up jobs joined with job-222 and disjoin all subjobs
>> created from job- 222
>> 2019-08-28 13:23:14,888 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) VM is already stopped: VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:23:14,888 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Done executing VM work job:
>>
>> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
>> 2019-08-28 13:23:14,888 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Complete async job-223, jobStatus: SUCCEEDED, resultCode:
>> 0, result: null
>> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Publish async job-223 complete on message bus
>> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Wake up jobs related to job-223
>> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Update db status for job-223
>> 2019-08-28 13:23:14,892 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
>> (logid:3e502c4b) Wake up jobs joined with job-223 and disjoin all subjobs
>> created from job- 223
>> 2019-08-28 13:23:14,895 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
>> with run of VM work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
>> 2019-08-28 13:23:14,895 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
>> executing com.cloud.vm.VmWorkStop for job-222
>> 2019-08-28 13:23:14,898 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
>> Remove
>> job-222 from job monitoring
>> 2019-08-28 13:23:14,901 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
>> with run of VM work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
>> 2019-08-28 13:23:14,901 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
>> executing com.cloud.vm.VmWorkStop for job-223
>> 2019-08-28 13:23:14,903 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
>> Remove
>> job-223 from job monitoring
>> 2019-08-28 13:23:14,916 DEBUG [c.c.c.CapacityManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) VM state transitted from
>> :Stopped to Expunging with event: ExpungeOperationvm's original host id:
>> null new host id: null host id before state transition: null
>> 2019-08-28 13:23:14,921 DEBUG [c.c.c.CapacityManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) VM state transitted from
>> :Stopped to Expunging with event: ExpungeOperationvm's original host id:
>> null new host id: null host id before state transition: null
>> 2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Destroying vm
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up NICS
>> 2019-08-28 13:23:14,923 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning network for vm: 67
>> 2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Destroying vm
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up NICS
>> 2019-08-28 13:23:14,929 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning network for vm: 68
>> 2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking VirtualRouter to
>> release Nic[184-67-null-137.74.82.208]
>> 2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking SecurityGroupProvider
>> to release Nic[184-67-null-137.74.82.208]
>> 2019-08-28 13:23:14,936 DEBUG [c.c.n.g.DirectNetworkGuru]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Deallocate network:
>> networkId:
>> 203, ip: 137.74.82.208
>> 2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking VirtualRouter to
>> release Nic[187-68-null-137.74.82.209]
>> 2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking
>> SecurityGroupProvider
>> to release Nic[187-68-null-137.74.82.209]
>> 2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Deallocate network:
>> networkId: 203, ip: 137.74.82.209
>> 2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) remove nic 184 secondary ip
>> 2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=184
>> 2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
>> entry ...
>> 2019-08-28 13:23:14,943 DEBUG [c.c.n.g.DirectNetworkGuru]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) remove nic 187 secondary ip
>> 2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=187
>> 2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
>> entry ...
>> 2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=185
>> 2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
>> entry ...
>> 2019-08-28 13:23:14,949 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=188
>> 2019-08-28 13:23:14,950 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
>> entry ...
>> 2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=186
>> 2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
>> entry ...
>> 2019-08-28 13:23:14,952 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up hypervisor data
>> structures (ex. SRs in XenServer) for managed storage
>> 2019-08-28 13:23:14,952 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning storage for vm: 67
>> 2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=189
>> 2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
>> entry ...
>> 2019-08-28 13:23:14,953 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up hypervisor data
>> structures (ex. SRs in XenServer) for managed storage
>> 2019-08-28 13:23:14,954 DEBUG [o.a.c.e.o.VolumeOrchestrator]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning storage for vm: 68
>> 2019-08-28 13:23:14,964 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) getCommandHostDelegation:
>> class org.apache.cloudstack.storage.command.DeleteCommand
>> 2019-08-28 13:23:14,964 DEBUG [c.c.h.XenServerGuru]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) We are returning the default
>> host to execute commands because the command is not of Copy type.
>> 2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
>> (secstorage-1:ctx-59a5da7a)
>> (logid:2183e74c) Seq 1-5778962746846478725: Sending { Cmd , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>>
>> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
>> }
>> 2019-08-28 13:23:14,965 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) getCommandHostDelegation:
>> class org.apache.cloudstack.storage.command.DeleteCommand
>> 2019-08-28 13:23:14,965 DEBUG [c.c.h.XenServerGuru]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) We are returning the
>> default
>> host to execute commands because the command is not of Copy type.
>> 2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
>> Sending { Cmd , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver:
>> v1, Flags: 100011,
>>
>> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.
>> cloudstack.storage.to
>> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
>> apache.cloudstack.storage.to
>> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
>> }
>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-2:null) (logid:) Seq 1-5778962746846478726:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-1:null) (logid:) Seq 1-5778962746846478725:
>> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
>> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
>> Ver: v1, Flags: 10, { Answer } }
>> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
>> (secstorage-1:ctx-59a5da7a)
>> (logid:2183e74c) Seq 1-5778962746846478725: Received: { Ans: , MgmtId:
>> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
>> Answer }
>> }
>> 2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Volume 63 is not referred
>> anywhere, remove it from volumes table
>> 2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Volume 62 is not referred
>> anywhere, remove it from volumes table
>> 2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expunged
>> VM[ConsoleProxy|v-68-VM]
>> 2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Expunged
>> VM[SecondaryStorageVm|s-67-VM]
>> 2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) received secondary storage vm
>> alert
>> 2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Secondary Storage Vm creation
>> failure, zone: France
>> 2019-08-28 13:23:14,975 WARN [c.c.a.AlertManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) AlertType:: 19 |
>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
>> Storage Vm creation failure. zone: France, error details: null
>> 2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) received console proxy
>> alert
>> 2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Console proxy creation
>> failure, zone: France
>> 2019-08-28 13:23:14,976 WARN [c.c.a.AlertManagerImpl]
>> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) AlertType:: 10 |
>> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
>> proxy creation failure. zone: France, error details: null
>> 2019-08-28 13:23:14,977 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
>> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Primary secondary storage is
>> not even started, wait until next turn
>> 2019-08-28 13:23:16,259 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-bd2c1e74) (logid:dc193f19) ===START===
>> 137.74.35.102
>> -- GET command=listInfrastructure&response=json&_=1566988128979
>> 2019-08-28 13:23:16,268 DEBUG [c.c.a.ApiServer]
>> (qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) CIDRs from
>> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is
>> allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2019-08-28 13:23:16,286 DEBUG [c.c.a.ApiServlet]
>> (qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) ===END===
>> 137.74.35.102 -- GET
>> command=listInfrastructure&response=json&_=1566988128979
>> 2019-08-28 13:23:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) Begin cleanup
>> expired async-jobs
>> 2019-08-28 13:23:24,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) End cleanup
>> expired
>> async-jobs
>> 2019-08-28 13:23:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
>> 2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-3:null) (logid:) Process host VM state report from
>> ping process. host: 1
>> 2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
>> number of records in report: 0
>> 2019-08-28 13:23:24,958 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
>> report. host: 1
>>
>
Re: Unable to start System VMs
Posted by Fariborz Navidan <md...@gmail.com>.
Thanks. I changed VLAN ID and now they are kept starting for along time.
Latest logs:
2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin cleanup
expired async-jobs
2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End cleanup expired
async-jobs
2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been pending for
959 seconds
2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been pending for
357 seconds
2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin cleanup
expired async-jobs
2019-08-28 14:34:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 routers to
update status.
2019-08-28 14:34:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC networks
to update Redundant State.
2019-08-28 14:34:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 networks to
update RvR status.
2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End cleanup expired
async-jobs
2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598229
2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598229
2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-e1f47dc0) (logid:c0c93cd1) Resetting hosts suitable for reconnect
2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed resetting hosts suitable for
reconnect
2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring hosts for clusters already
owned by this management server
2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed acquiring hosts for clusters
already owned by this management server
2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-e1f47dc0) (logid:c0c93cd1) Acquiring hosts for clusters not owned
by any management server
2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-e1f47dc0) (logid:c0c93cd1) Completed acquiring hosts for clusters
not owned by any management server
2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566995598230
2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) ===END===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566995598230
2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin cleanup
expired async-jobs
2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End cleanup expired
async-jobs
2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 14:34:24,974 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) Process VM state report. host: 1,
number of records in report: 1
2019-08-28 14:34:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) VM state report. host: 1, vm id:
103, power state: PowerOn
2019-08-28 14:34:24,976 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
1, vm id: 103, power state: PowerOn
2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 103, postpone power-change report by resetting
power-change counters
2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) Run missing VM report. current
time: 1566995664980
2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
id: 101, power state: PowerReportMissing, last state update: 1566995544000
2019-08-28 14:34:24,980 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) vm id: 101 - time since last state
update(120980ms) has passed graceful period
2019-08-28 14:34:24,982 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
1, vm id: 101, power state: PowerReportMissing
2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 101, postpone power-change report by resetting
power-change counters
2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) Detected missing VM. host: 1, vm
id: 102, power state: PowerReportMissing, last state update: 1566995544000
2019-08-28 14:34:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) vm id: 102 - time since last state
update(120980ms) has passed graceful period
2019-08-28 14:34:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) VM state report is updated. host:
1, vm id: 102, power state: PowerReportMissing
2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-15:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 102, postpone power-change report by resetting
power-change counters
2019-08-28 14:34:24,987 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-15:null) (logid:) Done with process of VM state
report. host: 1
2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598231
2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598231
2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566995598232
2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) ===END===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566995598232
2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling Monitor is
running...
2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233
2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234
2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598235
2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer]
(qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) Api
Discovery plugin was unable to find an api by that name or process any apis
2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598235
2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin cleanup
expired async-jobs
2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End cleanup expired
async-jobs
2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) ===END===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236
2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) HostStatsCollector is
running...
2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request]
(StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq 1-5778962746846479041:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector]
(StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector is
running...
2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector]
(StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no secondary
storage VM for secondary storage host Secondary1
2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning the
default host to execute commands because the command is not of Copy type.
2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request]
(StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq 1-5778962746846479042:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237
2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238
2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598239
2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer]
(qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) Api
Discovery plugin was unable to find an api by that name or process any apis
2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598239
2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598240
2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598240
2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin cleanup
expired async-jobs
2019-08-28 14:34:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 routers to
update status.
2019-08-28 14:34:44,855 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC networks
to update Redundant State.
2019-08-28 14:34:44,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 networks to
update RvR status.
2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End cleanup expired
async-jobs
2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START===
137.74.35.102 -- GET
command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:48,215 WARN [c.c.a.d.ParamGenericValidationWorker]
(qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) Received
unknown parameters for command listSystemVms. Unknown parameters : listall
2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) ===END===
137.74.35.102 -- GET
command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241
2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START===
137.74.35.102 -- GET
command=listHosts&response=json&details=min&_=1566995598242
2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl]
(qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) >>>Searching
for hosts>>>
2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl]
(qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5)
>>>Generating Response>>>
2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) ===END===
137.74.35.102 -- GET
command=listHosts&response=json&details=min&_=1566995598242
2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin cleanup
expired async-jobs
2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End cleanup expired
async-jobs
2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243
2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244
2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598245
2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598245
2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin cleanup
expired async-jobs
2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End cleanup expired
async-jobs
2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566995598246
2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) ===END===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566995598246
2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247
2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248
2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598249
2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer]
(qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) Api
Discovery plugin was unable to find an api by that name or process any apis
2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598249
2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been pending for
1019 seconds
2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been pending for
417 seconds
2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) ===END===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250
2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl]
(ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started resource
counters recalculation periodic task.
2019-08-28 14:35:14,848 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
(ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External devices
are not used. Skipping external device usage collection
2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin cleanup
expired async-jobs
2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
(SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot scheduler.poll is
being called at 2019-08-28 12:35:14 GMT
2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End cleanup expired
async-jobs
2019-08-28 14:35:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 routers to
update status.
2019-08-28 14:35:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterMonitor-1:ctx-7cb475c8) (logid:cc3e421d) Found 0 running routers.
2019-08-28 14:35:14,858 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC networks
to update Redundant State.
2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
(SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots to be
executed at 2019-08-28 12:35:14 GMT
2019-08-28 14:35:14,858 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 networks to
update RvR status.
2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator]
(Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4) NetworkGarbageCollector
uses '600' seconds for GC interval.
2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl]
(HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage server
2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl]
(HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true, heartbeat:
Wed Aug 28 14:35:12 CEST 2019
2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251
2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252
2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598253
2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer]
(qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) Api
Discovery plugin was unable to find an api by that name or process any apis
2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566995598253
2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598254
2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer]
(qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566995598254
2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin cleanup
expired async-jobs
2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End cleanup expired
async-jobs
2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
(LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager is
running and getting the updates from LB providers and updating service
status
2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
(LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck Manager is
running and getting the updates from LB providers and updating service
status
2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-1:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 14:35:25,005 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) Process VM state report. host: 1,
number of records in report: 1
2019-08-28 14:35:25,006 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) VM state report. host: 1, vm id:
103, power state: PowerOn
2019-08-28 14:35:25,007 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) VM state report is updated. host: 1,
vm id: 103, power state: PowerOn
2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-1:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 103, postpone power-change report by resetting
power-change counters
2019-08-28 14:35:25,011 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) Run missing VM report. current time:
1566995725011
2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm id:
101, power state: PowerReportMissing, last state update: 1566995664000
2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) vm id: 101 - time since last state
update(61011ms) has not passed graceful period yet
2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) Detected missing VM. host: 1, vm id:
102, power state: PowerReportMissing, last state update: 1566995664000
2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) vm id: 102 - time since last state
update(61011ms) has not passed graceful period yet
2019-08-28 14:35:25,012 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-1:null) (logid:) Done with process of VM state
report. host: 1
2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START===
137.74.35.102 -- GET
command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:26,390 WARN [c.c.a.d.ParamGenericValidationWorker]
(qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) Received
unknown parameters for command listSystemVms. Unknown parameters : listall
2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) ===END===
137.74.35.102 -- GET
command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255
2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START===
137.74.35.102 -- GET
command=listHosts&response=json&details=min&_=1566995598256
2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer]
(qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl]
(qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) >>>Searching
for hosts>>>
2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl]
(qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb)
>>>Generating Response>>>
2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) ===END===
137.74.35.102 -- GET
command=listHosts&response=json&details=min&_=1566995598256
2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request]
(StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq 1-5778962746846479043:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetVolumeStatsAnswer } }
2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl]
(StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from executing
class com.cloud.agent.api.GetVolumeStatsCommand:
2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling Monitor is
running...
2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257
2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer]
(qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258
2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin cleanup
expired async-jobs
2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End cleanup expired
async-jobs
2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) HostStatsCollector is
running...
2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request]
(StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq 1-5778962746846479044:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector is
running...
2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no secondary
storage VM for secondary storage host Secondary1
2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning the
default host to execute commands because the command is not of Copy type.
2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request]
(StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq 1-5778962746846479045:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259
2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer]
(qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260
2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261
2019-08-28 14:35:44,845 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Checking if
any host reservation can be released ...
2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin cleanup
expired async-jobs
2019-08-28 14:35:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 routers to
update status.
2019-08-28 14:35:44,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC networks
to update Redundant State.
2019-08-28 14:35:44,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 networks to
update RvR status.
2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Cannot
release reservation, Found 3 VMs Running on host 1
2019-08-28 14:35:44,856 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) Done running
HostReservationReleaseChecker ...
2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End cleanup expired
async-jobs
2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity Checker
...
2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating system
capacity
2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram capacity
update
2019-08-28 14:35:44,896 DEBUG [c.c.c.CapacityManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 3 VMs on host 1
2019-08-28 14:35:44,904 DEBUG [c.c.c.CapacityManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Found 0 VM, not running on
host 1
2019-08-28 14:35:44,907 DEBUG [c.c.c.CapacityManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
capacity, host:1 usedCpuCore: 3 reservedCpuCore: 0
2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate cpu
capacity, host:1 usedCpu: 1500 reservedCpu: 0
2019-08-28 14:35:44,908 DEBUG [c.c.c.CapacityManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) No need to calibrate memory
capacity, host:1 usedMem: 1879048192 reservedMem: 0
2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing cpu/ram
capacity update
2019-08-28 14:35:44,910 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing storage capacity
update
2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Found storage pool Primary1
of type SharedMountPoint
2019-08-28 14:35:44,918 DEBUG [c.c.s.StorageManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Total over provisioned
capacity of the pool Primary1 id: 1 is 984895062016
2019-08-28 14:35:44,920 DEBUG [c.c.s.StorageManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Successfully set Capacity -
984895062016 for capacity type - 3 , DataCenterId - 1, HostOrPoolId - 1,
PodId 1
2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing storage
capacity update
2019-08-28 14:35:44,920 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity updates
for public ip and Vlans
2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Done capacity updates for
public ip and Vlans
2019-08-28 14:35:44,938 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing capacity updates
for private ip
2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Done executing capacity
updates for private ip
2019-08-28 14:35:44,944 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Done recalculating system
capacity
2019-08-28 14:35:44,982 DEBUG [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-305972aa) (logid:95391fc9) Done running Capacity
Checker ...
2019-08-28 14:35:49,922 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-7f369e38) (logid:88e96986) Resetting hosts suitable for reconnect
2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-7f369e38) (logid:88e96986) Completed resetting hosts suitable for
reconnect
2019-08-28 14:35:49,925 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-7f369e38) (logid:88e96986) Acquiring hosts for clusters already
owned by this management server
2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-7f369e38) (logid:88e96986) Completed acquiring hosts for clusters
already owned by this management server
2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-7f369e38) (logid:88e96986) Acquiring hosts for clusters not owned
by any management server
2019-08-28 14:35:49,926 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-7f369e38) (logid:88e96986) Completed acquiring hosts for clusters
not owned by any management server
2019-08-28 14:35:49,953 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-5e476ef2) (logid:be235fb1) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
2019-08-28 14:35:49,961 DEBUG [c.c.a.ApiServer]
(qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:50,014 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-549:ctx-5e476ef2 ctx-6c8ae13e) (logid:be235fb1) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598262
2019-08-28 14:35:50,633 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-3f36683f) (logid:7cc15823) ===START===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
2019-08-28 14:35:50,643 DEBUG [c.c.a.ApiServer]
(qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 14:35:50,698 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-535:ctx-3f36683f ctx-cf77b110) (logid:7cc15823) ===END===
137.74.35.102 -- GET
command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598263
2019-08-28 14:35:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) Begin cleanup
expired async-jobs
2019-08-28 14:35:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-697625ef) (logid:545f2600) End cleanup expired
async-jobs
2019-08-28 14:36:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) Begin cleanup
expired async-jobs
2019-08-28 14:36:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0a07f4c9) (logid:a32b421b) End cleanup expired
async-jobs
2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-318) has been pending for
1079 seconds
2019-08-28 14:36:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-6c737611) (logid:5dbd71f1) Task (job-320) has been pending for
477 seconds
2019-08-28 14:36:14,855 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) Begin cleanup
expired async-jobs
2019-08-28 14:36:14,855 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 routers to
update status.
2019-08-28 14:36:14,857 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 VPC networks
to update Redundant State.
2019-08-28 14:36:14,859 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-78190b56) (logid:75dfcd2c) Found 0 networks to
update RvR status.
2019-08-28 14:36:14,864 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-90061a14) (logid:d64d012e) End cleanup expired
async-jobs
2019-08-28 14:36:24,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) Begin cleanup
expired async-jobs
2019-08-28 14:36:24,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-2f779805) (logid:45a1c5bd) End cleanup expired
async-jobs
2019-08-28 14:36:24,972 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-7:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 14:36:24,972 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) Process VM state report. host: 1,
number of records in report: 1
2019-08-28 14:36:24,973 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) VM state report. host: 1, vm id:
103, power state: PowerOn
2019-08-28 14:36:24,975 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
vm id: 103, power state: PowerOn
2019-08-28 14:36:24,976 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 103, postpone power-change report by resetting
power-change counters
2019-08-28 14:36:24,978 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) Run missing VM report. current time:
1566995784978
2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm id:
101, power state: PowerReportMissing, last state update: 1566995664000
2019-08-28 14:36:24,979 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) vm id: 101 - time since last state
update(120978ms) has passed graceful period
2019-08-28 14:36:24,981 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
vm id: 101, power state: PowerReportMissing
2019-08-28 14:36:24,981 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 101, postpone power-change report by resetting
power-change counters
2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) Detected missing VM. host: 1, vm id:
102, power state: PowerReportMissing, last state update: 1566995664000
2019-08-28 14:36:24,983 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) vm id: 102 - time since last state
update(120978ms) has passed graceful period
2019-08-28 14:36:24,984 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) VM state report is updated. host: 1,
vm id: 102, power state: PowerReportMissing
2019-08-28 14:36:24,984 INFO [c.c.v.VirtualMachineManagerImpl]
(AgentManager-Handler-7:null) (logid:) There is pending job or HA tasks
working on the VM. vm id: 102, postpone power-change report by resetting
power-change counters
2019-08-28 14:36:24,986 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-7:null) (logid:) Done with process of VM state
report. host: 1
2019-08-28 14:36:30,181 DEBUG [c.c.s.StatsCollector]
(StatsCollector-6:ctx-9c8e3bed) (logid:9058cad8) AutoScaling Monitor is
running...
2019-08-28 14:36:34,848 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) Begin cleanup
expired async-jobs
2019-08-28 14:36:34,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-a4ba96bd) (logid:be688b96) End cleanup expired
async-jobs
2019-08-28 14:36:37,458 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) HostStatsCollector is
running...
2019-08-28 14:36:37,515 DEBUG [c.c.a.t.Request]
(StatsCollector-4:ctx-71c1f2b0) (logid:94e16990) Seq 1-5778962746846479046:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2019-08-28 14:36:38,661 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) StorageCollector is
running...
2019-08-28 14:36:38,671 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) There is no secondary
storage VM for secondary storage host Secondary1
2019-08-28 14:36:38,675 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 14:36:38,675 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) We are returning the
default host to execute commands because the command is not of Copy type.
2019-08-28 14:36:38,729 DEBUG [c.c.a.t.Request]
(StatsCollector-2:ctx-3dfe46e8) (logid:a3085aca) Seq 1-5778962746846479047:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
On Wed, Aug 28, 2019 at 4:04 PM Paul Angus <pa...@shapeblue.com> wrote:
> The error is here Fariborz,
>
> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
> create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
>
> you're already using a VLAN that you've also asked CloudStack to use.
>
>
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London WC2E 9DPUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: Fariborz Navidan <md...@gmail.com>
> Sent: 28 August 2019 12:29
> To: users@cloudstack.apache.org
> Subject: Unable to start System VMs
>
> Hello,
>
> New ACS management is unable to start System VMs. Below are logs. ACS is
> still poor in error handling and reporting. It should be able to retyrn
> clear error messages inside UI.
>
> 2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
> expired async-jobs
> 2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup expired
> async-jobs
> 2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
> expired async-jobs
> 2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup expired
> async-jobs
> 2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
> 2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) ===END===
> 137.74.35.102 -- GET
> command=listZones&page=1&pagesize=20&response=json&_=1566988128964
> 2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START=== 137.74.35.102
> -- GET
>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
> 2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) ===END===
> 137.74.35.102 -- GET
>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
> 2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
> 137.74.35.102 -- GET
>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
> 2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) ===END===
> 137.74.35.102 -- GET
>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
> 2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START=== 137.74.35.102
> -- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128967
> 2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api Discovery
> plugin was unable to find an api by that name or process any apis
> 2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) ===END===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566988128967
> 2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
> expired async-jobs
> 2019-08-28 13:22:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers to
> update status.
> 2019-08-28 13:22:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 13:22:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 networks to
> update RvR status.
> 2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup expired
> async-jobs
> 2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
> 137.74.35.102 -- GET
>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
> 2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) ===END===
> 137.74.35.102 -- GET
>
> command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
> 2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START=== 137.74.35.102
> -- GET
>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
> 2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) ===END===
> 137.74.35.102 -- GET
>
> command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
> 2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
> 137.74.35.102 -- GET
>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
> 2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) ===END===
> 137.74.35.102 -- GET
>
> command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
> 2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START=== 137.74.35.102
> -- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128971
> 2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api Discovery
> plugin was unable to find an api by that name or process any apis
> 2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) ===END===
> 137.74.35.102 -- GET
> command=listApis&response=json&name=listVmwareDcs&_=1566988128971
> 2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for reconnect
> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts suitable for
> reconnect
> 2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters already
> owned by this management server
> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for clusters
> already owned by this management server
> 2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not owned
> by any management server
> 2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for clusters
> not owned by any management server
> 2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128972
> 2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128972
> 2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
> expired async-jobs
> 2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup expired
> async-jobs
> 2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-3:null) (logid:) Process host VM state report from
> ping process. host: 1
> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
> number of records in report: 0
> 2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
> report. host: 1
> 2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor is
> running...
> 2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START=== 137.74.35.102
> -- GET command=listInfrastructure&response=json&_=1566988128973
> 2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128973
> 2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector is
> running...
> 2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
> (StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq 1-5778962746846478713:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
> running...
> 2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no secondary
> storage VM for secondary storage host Secondary1
> 2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (StatsCollector-3:ctx-07390103) (logid:8f414a20) getCommandHostDelegation:
> class com.cloud.agent.api.GetStorageStatsCommand
> 2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
> (StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
> default host to execute commands because the command is not of Copy type.
> 2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
> (StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq 1-5778962746846478714:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
> expired async-jobs
> 2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup expired
> async-jobs
> 2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to launch
> secondary storage VM
> 2019-08-28 13:22:40,562 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary storage
> vms found in datacenter id=1, starting one
> 2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary storage
> vm is available, need to allocate a new secondary storage vm
> 2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage vm
> from a newly started instance for request from data center : 1
> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> configuration for offering [Network Offering
> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
> 2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> configuration for offering [Network Offering
> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
> 2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
> configuration for offering [Network Offering
> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
> 2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to launch
> console proxy
> 2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Expand console proxy standby
> capacity for zone France
> 2019-08-28 13:22:40,615 INFO [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) No stopped console proxy is
> available, need to allocate a new console proxy
> 2019-08-28 13:22:40,616 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Assign console proxy from a
> newly started instance for request from data center : 1
> 2019-08-28 13:22:40,623 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating entries for VM:
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:22:40,625 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nics for
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:22:40,628 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[203|Guest|6] with requested
> profile NicProfile[0-0-null-null-null
> 2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
> configuration for offering [Network Offering
> [6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
> 2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> 2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
> configuration for offering [Network Offering
> [3-Control-System-Control-Network]: Ntwk[201|Control|3]
> 2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> 2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
> configuration for offering [Network Offering
> [2-Management-System-Management-Network]: Ntwk[200|Management|2]
> 2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
> Acct[193f925a-c974-11e9-b808-d05099d38681-system]
> 2019-08-28 13:22:40,660 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[201|Control|3] with
> requested profile null
> 2019-08-28 13:22:40,668 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
> VM[SecondaryStorageVm|s-67-VM] in network Ntwk[200|Management|2] with
> requested profile null
> 2019-08-28 13:22:40,673 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating entries for VM:
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:22:40,675 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nics for
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:22:40,676 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
> VM[ConsoleProxy|v-68-VM] in network Ntwk[203|Guest|6] with requested
> profile NicProfile[0-0-null-null-null
> 2019-08-28 13:22:40,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating disks for
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:22:40,681 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocation completed for VM:
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
> alert
> 2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) New secondary storage vm
> created, zone: France, secStorageVm: s-67-VM, public IP: null, private IP:
> null
> 2019-08-28 13:22:40,688 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
> VM[ConsoleProxy|v-68-VM] in network Ntwk[201|Control|3] with requested
> profile null
> 2019-08-28 13:22:40,690 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Sync job-218 execution on
> object VmWorkJobQueue.67
> 2019-08-28 13:22:40,691 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
> VM[ConsoleProxy|v-68-VM] in network Ntwk[200|Management|2] with requested
> profile null
> 2019-08-28 13:22:40,694 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating disks for
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:22:40,696 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocation completed for VM:
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy alert
> 2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) New console proxy created,
> zone: France, proxy: v-68-VM, public IP: null, private IP: null
> 2019-08-28 13:22:40,701 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Sync job-219 execution on
> object VmWorkJobQueue.68
> 2019-08-28 13:22:40,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
> item: SyncQueueItemVO {id:194, queueId: 191, contentType: AsyncJob,
> contentId: 218, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
> lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
> 13:22:40 CEST 2019}
> 2019-08-28 13:22:40,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
> job-218
> 2019-08-28 13:22:40,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
> item: SyncQueueItemVO {id:195, queueId: 192, contentType: AsyncJob,
> contentId: 219, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
> lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
> 13:22:40 CEST 2019}
> 2019-08-28 13:22:40,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
> job-219
> 2019-08-28 13:22:40,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:d5030cfc) Add
> job-218 into job monitoring
> 2019-08-28 13:22:40,872 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:8371f7e8) Add
> job-219 into job monitoring
> 2019-08-28 13:22:40,877 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
> Executing AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType: null,
> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
> null}
> 2019-08-28 13:22:40,877 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Run VM
> work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Execute VM work job:
>
> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:22:40,878 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
> Executing AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType: null,
> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
> null}
> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Run VM
> work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
> 2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Execute VM work job:
>
> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:22:40,880 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) VM state transitted from :Stopped to Starting with event:
> StartRequestedvm's original host id: null new host id: null host id before
> state transition: null
> 2019-08-28 13:22:40,880 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Successfully transitioned to start state for
> VM[SecondaryStorageVm|s-67-VM] reservation id =
> c608b88c-7d06-411e-bb84-f2d5e2b3390d
> 2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:22:40,881 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
> StartRequestedvm's original host id: null new host id: null host id before
> state transition: null
> 2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Successfully transitioned to start state for
> VM[ConsoleProxy|v-68-VM] reservation id =
> 6c836a9c-0d16-42c3-86c7-446febb1a189
> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
> pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
> 2019-08-28 13:22:40,882 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Searching all possible resources under this Zone: 1
> 2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Listing clusters in order of aggregate capacity, that have
> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
> pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
> 2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Searching all possible resources under this Zone: 1
> 2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
> 2019-08-28 13:22:40,886 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:22:40,886 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:22:40,887 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
> Ram=512
> 2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
> Ram=1024
> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
> capacity for requested CPU: 500 and requested RAM: 536870912 ,
> cpuOverprovisioningFactor: 4.0
> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
> and CPU after applying overprovisioning: 216000
> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
> CPU: 500
> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
> Requested RAM: 536870912
> 2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
> available
> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
> 216000; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
> 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding to
> list: 1
> 2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
> suitable hosts
> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) We need to allocate new storagepool for this volume
> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Calling StoragePoolAllocators to find suitable pools
> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
> 2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
> capacity for requested CPU: 500 and requested RAM: 1073741824 ,
> cpuOverprovisioningFactor: 4.0
> 2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) System VMs will use shared storage for zone id=1
> 2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) LocalStoragePoolAllocator trying to find storage pool to
> fit the vm
> 2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) ClusterScopeStoragePoolAllocator looking for storage pool
> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
> pools will be ignored.
> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Found pools matching tags: [Pool[1|SharedMountPoint]]
> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
> and CPU after applying overprovisioning: 216000
> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
> CPU: 500
> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
> Requested RAM: 1073741824
> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
> available
> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
> 216000; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
> 1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding to
> list: 1
> 2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
> suitable hosts
> 2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking if storage pool is suitable, name: null ,poolId:
> 1
> 2019-08-28 13:22:40,891 INFO [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Storage pool null (1) does not supply IOPS capacity,
> assuming enough capacity
> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) We need to allocate new storagepool for this volume
> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Calling StoragePoolAllocators to find suitable pools
> 2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking pool 1 for storage, totalSize: 984895062016,
> usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold:
> 0.85
> 2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Destination pool id: 1
> 2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) System VMs will use shared storage for zone id=1
> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) LocalStoragePoolAllocator trying to find storage pool to
> fit the vm
> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) ClusterScopeStoragePoolAllocator looking for storage pool
> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
> pools will be ignored.
> 2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Found pools matching tags: [Pool[1|SharedMountPoint]]
> 2019-08-28 13:22:40,893 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking if storage pool is suitable, name: null ,poolId:
> 1
> 2019-08-28 13:22:40,893 INFO [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Storage pool null (1) does not supply IOPS capacity,
> assuming enough capacity
> 2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking pool 1 for storage, totalSize: 984895062016,
> usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold:
> 0.85
> 2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Destination pool id: 1
> 2019-08-28 13:22:40,896 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Pool ID for the volume with ID 62 is null
> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Found storage pool Primary1 of type SharedMountPoint
> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Total capacity of the pool Primary1 with ID 1 is
> 984895062016
> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking pool: 1 for storage allocation , maxSize :
> 984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
> disable threshold: 0.85
> 2019-08-28 13:22:40,897 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) ClusterScopeStoragePoolAllocator returning 1 suitable
> storage pools
> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Trying to find a potenial host and associated storage
> pools from the suitable host/pool lists for this VM
> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking if host: 1 can access any suitable storage pool
> for volume: ROOT
> 2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Host: 1 can access pool: 1
> 2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Pool ID for the volume with ID 63 is null
> 2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Found storage pool Primary1 of type SharedMountPoint
> 2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Total capacity of the pool Primary1 with ID 1 is
> 984895062016
> 2019-08-28 13:22:40,898 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com and
> associated storage pools for this VM
> 2019-08-28 13:22:40,899 DEBUG [c.c.s.StorageManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking pool: 1 for storage allocation , maxSize :
> 984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
> disable threshold: 0.85
> 2019-08-28 13:22:40,899 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) ClusterScopeStoragePoolAllocator returning 1 suitable
> storage pools
> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Trying to find a potenial host and associated storage
> pools from the suitable host/pool lists for this VM
> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
> for volume: ROOT
> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Returning Deployment Destination:
>
> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
> 2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
>
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Host: 1 can access pool: 1
> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com and
> associated storage pools for this VM
> 2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Returning Deployment Destination:
>
> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
> 2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
>
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
> 2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) VM state transitted from :Starting to Starting with event:
> OperationRetryvm's original host id: null new host id: 1 host id before
> state transition: null
> 2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) VM state transitted from :Starting to Starting with event:
> OperationRetryvm's original host id: null new host id: 1 host id before
> state transition: null
> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) We are allocating VM, increasing the used capacity of this
> host:1
> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested RAM:
> 536870912
> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
> reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
> used:500, reserved:0; requested cpu:500,alloc_from_last:false
> 2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
> reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
> requested mem: 536870912,alloc_from_last:false
> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) We are allocating VM, increasing the used capacity of this
> host:1
> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU:
> 500
> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
> ,Requested RAM: 1073741824
> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
> old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
> new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
> 2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
> 536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
> reserved: 0; requested mem: 1073741824,alloc_from_last:false
> 2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Network id=203 is already implemented
> 2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Network id=203 is already implemented
> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Asking VirtualRouter to prepare for
> Nic[184-67-null-137.74.82.208]
> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Asking VirtualRouter to prepare for
> Nic[187-68-null-137.74.82.209]
> 2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Asking SecurityGroupProvider to prepare for
> Nic[184-67-null-137.74.82.208]
> 2019-08-28 13:22:40,913 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Asking SecurityGroupProvider to prepare for
> Nic[187-68-null-137.74.82.209]
> 2019-08-28 13:22:40,914 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Network id=201 is already implemented
> 2019-08-28 13:22:40,915 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Network id=201 is already implemented
> 2019-08-28 13:22:40,919 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Network id=200 is already implemented
> 2019-08-28 13:22:40,920 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Network id=200 is already implemented
> 2019-08-28 13:22:40,922 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Releasing ip address for instance=186
> 2019-08-28 13:22:40,923 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Releasing ip address for instance=189
> 2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Allocated a nic
> NicProfile[186-67-c608b88c-7d06-411e-bb84-f2d5e2b3390d-192.168.0.226-null
> for VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Allocated a nic
> NicProfile[189-68-6c836a9c-0d16-42c3-86c7-446febb1a189-192.168.0.121-null
> for VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) template 3 is already in store:1, type:Image
> 2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) template 3 is already in store:1, type:Image
> 2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) template 3 is already in store:1, type:Primary
> 2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) template 3 is already in store:1, type:Primary
> 2019-08-28 13:22:40,932 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) copyAsync inspecting src type TEMPLATE copyAsync
> inspecting dest type VOLUME
> 2019-08-28 13:22:40,933 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) copyAsync inspecting src type TEMPLATE copyAsync
> inspecting dest type VOLUME
> 2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) getCommandHostDelegation: class
> org.apache.cloudstack.storage.command.CopyCommand
> 2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) We are returning the default host to execute commands
> because the target hypervisor of the source data is not XenServer.
> 2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) getCommandHostDelegation: class
> org.apache.cloudstack.storage.command.CopyCommand
> 2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) We are returning the default host to execute commands
> because the target hypervisor of the source data is not XenServer.
> 2019-08-28 13:22:40,934 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Seq 1-5778962746846478715: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
>
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
> cloudstack.storage.to
> .TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> ","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
> Template
> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
> }
> 2019-08-28 13:22:40,935 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Seq 1-5778962746846478716: Waiting for Seq
> 5778962746846478715 Scheduling: { Cmd , MgmtId: 205425461278937, via: 1(
> fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
>
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
> cloudstack.storage.to
> .TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> ","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
> Template
> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
> }
> 2019-08-28 13:22:41,037 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-138d8f65) (logid:85352d35) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128974
> 2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478715:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 110,
>
> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
> }
> 2019-08-28 13:22:41,037 DEBUG [c.c.a.m.AgentAttache]
> (AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478716: Sending
> now. is current sequence.
> 2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Seq 1-5778962746846478715: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
> CopyCmdAnswer } }
> 2019-08-28 13:22:41,040 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:41,047 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128974
> 2019-08-28 13:22:41,057 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
> template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
> zone=1 pod=1 guid=s-67-VM workers=5
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
> dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
> 2019-08-28 13:22:41,071 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Seq 1-5778962746846478717: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
> dns1=8.8.8.8 dns2=8.8.4.4
>
> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
> }
> 2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 110,
>
> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
> }
> 2019-08-28 13:22:41,126 DEBUG [c.c.a.m.AgentAttache]
> (AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716: No more
> commands found
> 2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Seq 1-5778962746846478716: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
> CopyCmdAnswer } }
> 2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-9:null) (logid:) Seq 1-5778962746846478717:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
> dns1=8.8.8.8 dns2=8.8.4.4
>
> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
> to create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>
> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous failure","wait":0}}] }
> 2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Seq 1-5778962746846478717: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StartAnswer, Answer } }
> 2019-08-28 13:22:41,128 INFO [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
> create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
> 2019-08-28 13:22:41,129 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Cleaning up resources for the vm
> VM[SecondaryStorageVm|s-67-VM] in Starting state
> 2019-08-28 13:22:41,131 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Seq 1-5778962746846478718: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.64","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-08-28 13:22:41,139 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
> type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
> guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
> eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.2.191
> eth0mask=255.255.0.0 eth1ip=192.168.0.121 eth1mask=255.255.255.0 mgmtcidr=
> 51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
> internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
> 2019-08-28 13:22:41,151 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Seq 1-5778962746846478719: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>
> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.191","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
> }
> 2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-5778962746846478719:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>
> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
> to create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>
> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous failure","wait":0}}] }
> 2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Seq 1-5778962746846478719: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StartAnswer, Answer } }
> 2019-08-28 13:22:41,323 INFO [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
> create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
> 2019-08-28 13:22:41,326 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
> in Starting state
> 2019-08-28 13:22:41,337 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Seq 1-5778962746846478720: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.2.191","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-08-28 13:22:44,313 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-10:null) (logid:) Seq 1-5778962746846478718:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
> 2019-08-28 13:22:44,314 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Seq 1-5778962746846478718: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StopAnswer } }
> 2019-08-28 13:22:44,318 DEBUG [c.c.n.g.ControlNetworkGuru]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
> 2019-08-28 13:22:44,321 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Releasing ip address for
> reservationId=c608b88c-7d06-411e-bb84-f2d5e2b3390d, instance=186
> 2019-08-28 13:22:44,321 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
> 2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Successfully released network resources for the vm
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Successfully cleaned up resources for the VM
> VM[SecondaryStorageVm|s-67-VM] in Starting state
> 2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
> 2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
> 2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
> 2019-08-28 13:22:44,324 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
> 2019-08-28 13:22:44,324 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Searching resources only under specified Cluster: 1
> 2019-08-28 13:22:44,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:22:44,327 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:22:44,328 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
> Ram=512
> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
> hostId: 1 is in avoid set, skipping this and trying other available hosts
> 2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
> suitable hosts
> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) No suitable hosts found
> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) No suitable hosts found under this Cluster: 1
> 2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Could not find suitable Deployment Destination for this VM
> under any clusters, returning.
> 2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Searching resources only under specified Cluster: 1
> 2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) The specified cluster is in avoid set, returning.
> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
> pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
> 2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
> 2019-08-28 13:22:44,331 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Searching all possible resources under this Zone: 1
> 2019-08-28 13:22:44,332 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Listing clusters in order of aggregate capacity, that have
> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
> 2019-08-28 13:22:44,333 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Removing from the clusterId list these clusters from avoid
> set: [1]
> 2019-08-28 13:22:44,334 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) No clusters found after removing disabled clusters and
> clusters in avoid list, returning.
> 2019-08-28 13:22:44,338 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) VM state transitted from :Starting to Stopped with event:
> OperationFailedvm's original host id: null new host id: null host id before
> state transition: 1
> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
> applying overprovisioning: 66088833024
> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
> actual total: 54000, total with overprovisioning: 216000; new used:
> 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
> 0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2019-08-28 13:22:44,343 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Invocation exception, caused by:
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:22:44,343 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
> (logid:e3068c13) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:22:44,343 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
> 2019-08-28 13:22:44,344 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Unable
> to complete AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
> null}, job origin:76
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:22:44,344 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
> Complete async job-218, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
> Publish async job-218 complete on message bus
> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
> up jobs related to job-218
> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Update
> db status for job-218
> 2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
> up jobs joined with job-218 and disjoin all subjobs created from job- 218
> 2019-08-28 13:22:44,349 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
> executing com.cloud.vm.VmWorkStart for job-218
> 2019-08-28 13:22:44,350 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Remove
> job-218 from job monitoring
> 2019-08-28 13:22:44,351 WARN [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Exception while trying to
> start secondary storage vm
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:22:44,352 INFO [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Unable to start secondary
> storage vm for standby capacity, vm id : 67, will recycle it and start a
> new one
> 2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
> alert
> 2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Secondary Storage Vm creation
> failure, zone: France
> 2019-08-28 13:22:44,352 WARN [c.c.a.AlertManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) AlertType:: 19 |
> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
> Storage Vm creation failure. zone: France, error details: null
> 2019-08-28 13:22:44,353 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
> (secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Primary secondary storage is
> not even started, wait until next turn
> 2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-11:null) (logid:) Seq 1-5778962746846478720:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
> 2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Seq 1-5778962746846478720: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StopAnswer } }
> 2019-08-28 13:22:44,436 DEBUG [c.c.n.g.ControlNetworkGuru]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
> 2019-08-28 13:22:44,438 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Releasing ip address for
> reservationId=6c836a9c-0d16-42c3-86c7-446febb1a189, instance=189
> 2019-08-28 13:22:44,438 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
> 2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Successfully released network resources for the vm
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Successfully cleaned up resources for the VM
> VM[ConsoleProxy|v-68-VM] in Starting state
> 2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
> 2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
> 2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
> 2019-08-28 13:22:44,441 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Searching resources only under specified Cluster: 1
> 2019-08-28 13:22:44,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:22:44,443 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
> Ram=1024
> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
> hostId: 1 is in avoid set, skipping this and trying other available hosts
> 2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
> suitable hosts
> 2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) No suitable hosts found
> 2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) No suitable hosts found under this Cluster: 1
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Could not find suitable Deployment Destination for this VM
> under any clusters, returning.
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Searching resources only under specified Cluster: 1
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) The specified cluster is in avoid set, returning.
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
> pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
> 2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
> 2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Searching all possible resources under this Zone: 1
> 2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
> (atleast one host with) enough CPU and RAM capacity under this Zone: 1
> 2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Removing from the clusterId list these clusters from avoid
> set: [1]
> 2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) No clusters found after removing disabled clusters and
> clusters in avoid list, returning.
> 2019-08-28 13:22:44,450 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
> OperationFailedvm's original host id: null new host id: null host id before
> state transition: 1
> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
> applying overprovisioning: 66088833024
> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
> actual total: 54000, total with overprovisioning: 216000; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
> 0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Invocation exception, caused by:
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:22:44,453 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
> (logid:3e502c4b) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:22:44,453 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
> 2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Unable
> to complete AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
> null}, job origin:77
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
> Complete async job-219, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
> Publish async job-219 complete on message bus
> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
> up jobs related to job-219
> 2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Update
> db status for job-219
> 2019-08-28 13:22:44,455 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
> up jobs joined with job-219 and disjoin all subjobs created from job- 219
> 2019-08-28 13:22:44,457 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
> executing com.cloud.vm.VmWorkStart for job-219
> 2019-08-28 13:22:44,458 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Remove
> job-219 from job monitoring
> 2019-08-28 13:22:44,459 WARN [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Exception while trying to
> start console proxy
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:22:44,460 INFO [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Unable to start console
> proxy vm for standby capacity, vm id : 68, will recycle it and start a new
> one
> 2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy alert
> 2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Console proxy creation
> failure, zone: France
> 2019-08-28 13:22:44,460 WARN [c.c.a.AlertManagerImpl]
> (consoleproxy-1:ctx-71d4f124) (logid:84bfc356) AlertType:: 10 |
> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
> proxy creation failure. zone: France, error details: null
> 2019-08-28 13:22:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) Begin cleanup
> expired async-jobs
> 2019-08-28 13:22:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 routers to
> update status.
> 2019-08-28 13:22:44,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 13:22:44,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 networks to
> update RvR status.
> 2019-08-28 13:22:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) End cleanup expired
> async-jobs
> 2019-08-28 13:22:48,876 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-1ca6b8a4) (logid:767cda38) ===START=== 137.74.35.102
> -- GET command=listInfrastructure&response=json&_=1566988128975
> 2019-08-28 13:22:48,883 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:48,903 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128975
> 2019-08-28 13:22:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) Begin cleanup
> expired async-jobs
> 2019-08-28 13:22:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) End cleanup expired
> async-jobs
> 2019-08-28 13:22:56,741 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-3fa512ab) (logid:4609e172) ===START===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128976
> 2019-08-28 13:22:56,747 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:22:56,751 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128976
> 2019-08-28 13:23:03,063 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-a0c69e67) (logid:0a2b6c35) ===START=== 137.74.35.102
> -- GET command=listInfrastructure&response=json&_=1566988128977
> 2019-08-28 13:23:03,071 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:23:03,091 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128977
> 2019-08-28 13:23:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) Begin cleanup
> expired async-jobs
> 2019-08-28 13:23:04,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) End cleanup expired
> async-jobs
> 2019-08-28 13:23:09,900 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-16:ctx-9f3738f2) (logid:dea95b1b) ===START=== 137.74.35.102
> -- GET command=listInfrastructure&response=json&_=1566988128978
> 2019-08-28 13:23:09,907 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:23:09,929 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128978
> 2019-08-28 13:23:10,557 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Zone 1 is ready to launch
> secondary storage VM
> 2019-08-28 13:23:10,561 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) No running secondary storage
> vms found in datacenter id=1, starting one
> 2019-08-28 13:23:10,570 INFO [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Found a stopped secondary
> storage vm, starting it. Vm id : 67
> 2019-08-28 13:23:10,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-220 execution on
> object VmWorkJobQueue.67
> 2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Zone 1 is ready to launch
> console proxy
> 2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expand console proxy standby
> capacity for zone France
> 2019-08-28 13:23:10,616 INFO [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Found a stopped console
> proxy, starting it. Vm id : 68
> 2019-08-28 13:23:10,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-221 execution on
> object VmWorkJobQueue.68
> 2019-08-28 13:23:10,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
> item: SyncQueueItemVO {id:196, queueId: 191, contentType: AsyncJob,
> contentId: 220, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
> lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
> 13:23:10 CEST 2019}
> 2019-08-28 13:23:10,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
> job-220
> 2019-08-28 13:23:10,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
> item: SyncQueueItemVO {id:197, queueId: 192, contentType: AsyncJob,
> contentId: 221, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
> lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
> 13:23:10 CEST 2019}
> 2019-08-28 13:23:10,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
> job-221
> 2019-08-28 13:23:10,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:f78794fa) Add
> job-220 into job monitoring
> 2019-08-28 13:23:10,871 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:7deee6cf) Add
> job-221 into job monitoring
> 2019-08-28 13:23:10,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
> Executing AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType: null,
> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
> null}
> 2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Run VM
> work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
> 2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Execute VM work job:
>
> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:23:10,881 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
> Executing AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType: null,
> instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
> null}
> 2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Run VM
> work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
> 2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Execute VM work job:
>
> com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:23:10,883 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) VM state transitted from :Stopped to Starting with event:
> StartRequestedvm's original host id: null new host id: null host id before
> state transition: null
> 2019-08-28 13:23:10,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Successfully transitioned to start state for
> VM[SecondaryStorageVm|s-67-VM] reservation id =
> 46a0e954-a9af-45af-ac39-91e036654f95
> 2019-08-28 13:23:10,884 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
> StartRequestedvm's original host id: null new host id: null host id before
> state transition: null
> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Successfully transitioned to start state for
> VM[ConsoleProxy|v-68-VM] reservation id =
> a25c0118-b49c-41b2-8970-9b73191746f3
> 2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
> 2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Searching resources only under specified Cluster: 1
> 2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
> 2019-08-28 13:23:10,886 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Searching resources only under specified Cluster: 1
> 2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
> Ram=1024
> 2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
> Ram=512
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
> (cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
> capacity for requested CPU: 500 and requested RAM: 1073741824 ,
> cpuOverprovisioningFactor: 4.0
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
> capacity for requested CPU: 500 and requested RAM: 536870912 ,
> cpuOverprovisioningFactor: 4.0
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
> and CPU after applying overprovisioning: 216000
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
> CPU: 500
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
> Requested RAM: 1073741824
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
> available
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
> 216000; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
> 1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding to
> list: 1
> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
> suitable hosts
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
> and CPU after applying overprovisioning: 216000
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
> CPU: 500
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
> Requested RAM: 536870912
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
> available
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
> 1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
> 216000; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
> 1, used: 0, reserved: 0, total: 66088833024; requested mem:
> 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding to
> list: 1
> 2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
> suitable hosts
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Volume has pool already allocated, checking if pool can be
> reused, poolId: 1
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Volume has pool already allocated, checking if pool can be
> reused, poolId: 1
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Planner need not allocate a pool for this volume since its
> READY
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Trying to find a potenial host and associated storage
> pools from the suitable host/pool lists for this VM
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
> for volume: ROOT
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Planner need not allocate a pool for this volume since its
> READY
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Trying to find a potenial host and associated storage
> pools from the suitable host/pool lists for this VM
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Checking if host: 1 can access any suitable storage pool
> for volume: ROOT
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Host: 1 can access pool: 1
> 2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Host: 1 can access pool: 1
> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com and
> associated storage pools for this VM
> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com and
> associated storage pools for this VM
> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Returning Deployment Destination:
>
> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
>
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Returning Deployment Destination:
>
> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
>
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) VM state transitted from :Starting to Starting with event:
> OperationRetryvm's original host id: null new host id: 1 host id before
> state transition: null
> 2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) VM state transitted from :Starting to Starting with event:
> OperationRetryvm's original host id: null new host id: 1 host id before
> state transition: null
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) We are allocating VM, increasing the used capacity of this
> host:1
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested RAM:
> 536870912
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
> reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
> used:500, reserved:0; requested cpu:500,alloc_from_last:false
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
> reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
> requested mem: 536870912,alloc_from_last:false
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) We are allocating VM, increasing the used capacity of this
> host:1
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU:
> 500
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
> ,Requested RAM: 1073741824
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
> old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
> new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
> 2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
> 536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
> reserved: 0; requested mem: 1073741824,alloc_from_last:false
> 2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Network id=203 is already implemented
> 2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Network id=203 is already implemented
> 2019-08-28 13:23:10,907 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Asking VirtualRouter to prepare for
> Nic[187-68-null-137.74.82.209]
> 2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Asking VirtualRouter to prepare for
> Nic[184-67-null-137.74.82.208]
> 2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Asking SecurityGroupProvider to prepare for
> Nic[187-68-null-137.74.82.209]
> 2019-08-28 13:23:10,909 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Asking SecurityGroupProvider to prepare for
> Nic[184-67-null-137.74.82.208]
> 2019-08-28 13:23:10,911 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Network id=201 is already implemented
> 2019-08-28 13:23:10,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Network id=201 is already implemented
> 2019-08-28 13:23:10,916 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Network id=200 is already implemented
> 2019-08-28 13:23:10,918 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Network id=200 is already implemented
> 2019-08-28 13:23:10,918 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Releasing ip address for instance=186
> 2019-08-28 13:23:10,920 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Allocated a nic
> NicProfile[186-67-46a0e954-a9af-45af-ac39-91e036654f95-192.168.0.86-null
> for VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:23:10,921 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Releasing ip address for instance=189
> 2019-08-28 13:23:10,922 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) No need to recreate the volume: Vol[62|vm=67|ROOT], since
> it already has a pool assigned: 1, adding disk to VM
> 2019-08-28 13:23:10,923 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Allocated a nic
> NicProfile[189-68-a25c0118-b49c-41b2-8970-9b73191746f3-192.168.0.20-null
> for VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:23:10,925 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) No need to recreate the volume: Vol[63|vm=68|ROOT], since
> it already has a pool assigned: 1, adding disk to VM
> 2019-08-28 13:23:10,930 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
> template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
> zone=1 pod=1 guid=s-67-VM workers=5
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
> dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
> 2019-08-28 13:23:10,931 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
> type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
> guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
> eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.0.248
> eth0mask=255.255.0.0 eth1ip=192.168.0.20 eth1mask=255.255.255.0 mgmtcidr=
> 51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
> internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
> 2019-08-28 13:23:10,941 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Seq 1-5778962746846478721: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
> dns1=8.8.8.8 dns2=8.8.4.4
>
> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.170","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
> }
> 2019-08-28 13:23:10,942 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Seq 1-5778962746846478722: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>
> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.248","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
> }
> 2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-12:null) (logid:) Seq 1-5778962746846478722:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
> name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
> eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
>
> dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
> to create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>
> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous failure","wait":0}}] }
> 2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Seq 1-5778962746846478722: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StartAnswer, Answer } }
> 2019-08-28 13:23:10,985 INFO [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
> create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
> 2019-08-28 13:23:10,986 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
> in Starting state
> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Seq 1-5778962746846478723: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.0.248","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-13:null) (logid:) Seq 1-5778962746846478721:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
>
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> 5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
> name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
> eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
> eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
> eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
> private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
> dns1=8.8.8.8 dns2=8.8.4.4
>
> nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
> to create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
>
> breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous failure","wait":0}}] }
> 2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Seq 1-5778962746846478721: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StartAnswer, Answer } }
> 2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
> create vnet 100: device eth1.100 is already a member of a bridge; can't
> enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
> 2019-08-28 13:23:10,990 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Cleaning up resources for the vm
> VM[SecondaryStorageVm|s-67-VM] in Starting state
> 2019-08-28 13:23:10,991 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Seq 1-5778962746846478724: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.170","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-08-28 13:23:14,130 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-14:null) (logid:) Seq 1-5778962746846478724:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
> 2019-08-28 13:23:14,131 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Seq 1-5778962746846478724: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StopAnswer } }
> 2019-08-28 13:23:14,136 DEBUG [c.c.n.g.ControlNetworkGuru]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
> 2019-08-28 13:23:14,138 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Releasing ip address for
> reservationId=46a0e954-a9af-45af-ac39-91e036654f95, instance=186
> 2019-08-28 13:23:14,139 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
> 2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Successfully released network resources for the vm
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Successfully cleaned up resources for the VM
> VM[SecondaryStorageVm|s-67-VM] in Starting state
> 2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
> 2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
> 2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
> 2019-08-28 13:23:14,142 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Searching resources only under specified Cluster: 1
> 2019-08-28 13:23:14,145 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:23:14,145 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:23:14,146 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz,
> Ram=512
> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
> hostId: 1 is in avoid set, skipping this and trying other available hosts
> 2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
> FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
> suitable hosts
> 2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) No suitable hosts found
> 2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) No suitable hosts found under this Cluster: 1
> 2019-08-28 13:23:14,148 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Could not find suitable Deployment Destination for this VM
> under any clusters, returning.
> 2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Searching resources only under specified Cluster: 1
> 2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) The specified cluster is in avoid set, returning.
> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:null, requested cpu: 500, requested ram: 536870912
> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
> 2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
> 2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Searching resources only under specified Pod: 1
> 2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Listing clusters in order of aggregate capacity, that have
> (atleast one host with) enough CPU and RAM capacity under this Pod: 1
> 2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Removing from the clusterId list these clusters from avoid
> set: [1]
> 2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) No clusters found after removing disabled clusters and
> clusters in avoid list, returning.
> 2019-08-28 13:23:14,156 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) VM state transitted from :Starting to Stopped with event:
> OperationFailedvm's original host id: null new host id: null host id before
> state transition: 1
> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
> applying overprovisioning: 66088833024
> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
> actual total: 54000, total with overprovisioning: 216000; new used:
> 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
> 0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2019-08-28 13:23:14,160 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Invocation exception, caused by:
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:23:14,161 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
> (logid:e3068c13) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:23:14,161 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
> 2019-08-28 13:23:14,161 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Unable
> to complete AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
> null}, job origin:76
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:23:14,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
> Complete async job-220, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
> Publish async job-220 complete on message bus
> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
> up jobs related to job-220
> 2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Update
> db status for job-220
> 2019-08-28 13:23:14,163 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
> up jobs joined with job-220 and disjoin all subjobs created from job- 220
> 2019-08-28 13:23:14,166 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
> executing com.cloud.vm.VmWorkStart for job-220
> 2019-08-28 13:23:14,166 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Remove
> job-220 from job monitoring
> 2019-08-28 13:23:14,169 WARN [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Exception while trying to
> start secondary storage vm
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:23:14,169 INFO [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Unable to start secondary
> storage vm for standby capacity, vm id : 67, will recycle it and start a
> new one
> 2019-08-28 13:23:14,172 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-222 execution on
> object VmWorkJobQueue.67
> 2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-15:null) (logid:) Seq 1-5778962746846478723:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
> 2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Seq 1-5778962746846478723: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
> StopAnswer } }
> 2019-08-28 13:23:14,623 DEBUG [c.c.n.g.ControlNetworkGuru]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
> 2019-08-28 13:23:14,626 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Releasing ip address for
> reservationId=a25c0118-b49c-41b2-8970-9b73191746f3, instance=189
> 2019-08-28 13:23:14,626 DEBUG [c.c.n.g.PodBasedNetworkGuru]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
> 2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Successfully released network resources for the vm
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Successfully cleaned up resources for the VM
> VM[ConsoleProxy|v-68-VM] in Starting state
> 2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
> 2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
> use this pool's dcId: 1 , podId: 1 , and clusterId: 1
> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
> 2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
> 2019-08-28 13:23:14,631 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Searching resources only under specified Cluster: 1
> 2019-08-28 13:23:14,635 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
> 2019-08-28 13:23:14,635 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
> pod:1 cluster:1
> 2019-08-28 13:23:14,636 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
> check for allocation: [Host[-1-Routing]]
> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
> Ram=1024
> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
> hostId: 1 is in avoid set, skipping this and trying other available hosts
> 2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
> FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
> suitable hosts
> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) No suitable hosts found
> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) No suitable hosts found under this Cluster: 1
> 2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Could not find suitable Deployment Destination for this VM
> under any clusters, returning.
> 2019-08-28 13:23:14,638 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Searching resources only under specified Cluster: 1
> 2019-08-28 13:23:14,639 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) The specified cluster is in avoid set, returning.
> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) DeploymentPlanner allocation algorithm: null
> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:null, requested cpu: 500, requested ram: 1073741824
> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
> 2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
> 2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Searching resources only under specified Pod: 1
> 2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
> (atleast one host with) enough CPU and RAM capacity under this Pod: 1
> 2019-08-28 13:23:14,641 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Removing from the clusterId list these clusters from avoid
> set: [1]
> 2019-08-28 13:23:14,642 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) No clusters found after removing disabled clusters and
> clusters in avoid list, returning.
> 2019-08-28 13:23:14,646 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
> OperationFailedvm's original host id: null new host id: null host id before
> state transition: 1
> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
> overprovisioning: 216000
> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
> applying overprovisioning: 66088833024
> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
> actual total: 54000, total with overprovisioning: 216000; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
> 0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2019-08-28 13:23:14,650 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Invocation exception, caused by:
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:23:14,651 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
> (logid:3e502c4b) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2019-08-28 13:23:14,651 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
> 2019-08-28 13:23:14,651 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Unable
> to complete AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
> null}, job origin:77
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
> Complete async job-221, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
> Publish async job-221 complete on message bus
> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
> up jobs related to job-221
> 2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Update
> db status for job-221
> 2019-08-28 13:23:14,653 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
> up jobs joined with job-221 and disjoin all subjobs created from job- 221
> 2019-08-28 13:23:14,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
> executing com.cloud.vm.VmWorkStart for job-221
> 2019-08-28 13:23:14,656 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Remove
> job-221 from job monitoring
> 2019-08-28 13:23:14,658 WARN [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Exception while trying to
> start console proxy
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
> at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-08-28 13:23:14,658 INFO [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Unable to start console
> proxy vm for standby capacity, vm id : 68, will recycle it and start a new
> one
> 2019-08-28 13:23:14,662 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-223 execution on
> object VmWorkJobQueue.68
> 2019-08-28 13:23:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 routers to
> update status.
> 2019-08-28 13:23:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 VPC networks
> to update Redundant State.
> 2019-08-28 13:23:14,855 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
> item: SyncQueueItemVO {id:198, queueId: 191, contentType: AsyncJob,
> contentId: 222, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
> lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
> 13:23:14 CEST 2019}
> 2019-08-28 13:23:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 networks to
> update RvR status.
> 2019-08-28 13:23:14,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
> job-222
> 2019-08-28 13:23:14,859 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
> item: SyncQueueItemVO {id:199, queueId: 192, contentType: AsyncJob,
> contentId: 223, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
> lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
> 13:23:14 CEST 2019}
> 2019-08-28 13:23:14,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
> job-223
> 2019-08-28 13:23:14,863 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:7e4a2784) Add
> job-222 into job monitoring
> 2019-08-28 13:23:14,869 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:26933ca4) Add
> job-223 into job monitoring
> 2019-08-28 13:23:14,871 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) Begin cleanup
> expired async-jobs
> 2019-08-28 13:23:14,875 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
> Executing AsyncJobVO {id:222, userId: 1, accountId: 1, instanceType: null,
> instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
>
> rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
> null}
> 2019-08-28 13:23:14,875 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Run VM
> work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
> 2019-08-28 13:23:14,877 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Execute VM work job:
>
> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:23:14,879 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) End cleanup expired
> async-jobs
> 2019-08-28 13:23:14,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
> Executing AsyncJobVO {id:223, userId: 1, accountId: 1, instanceType: null,
> instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
>
> rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
> null}
> 2019-08-28 13:23:14,881 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Run VM
> work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
> 2019-08-28 13:23:14,882 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Execute VM work job:
>
> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:23:14,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) VM is already stopped: VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:23:14,883 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Done executing VM work job:
>
> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:23:14,883 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Complete async job-222, jobStatus: SUCCEEDED, resultCode:
> 0, result: null
> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Publish async job-222 complete on message bus
> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Wake up jobs related to job-222
> 2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Update db status for job-222
> 2019-08-28 13:23:14,886 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
> (logid:e3068c13) Wake up jobs joined with job-222 and disjoin all subjobs
> created from job- 222
> 2019-08-28 13:23:14,888 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) VM is already stopped: VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:23:14,888 DEBUG [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Done executing VM work job:
>
> com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
> 2019-08-28 13:23:14,888 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Complete async job-223, jobStatus: SUCCEEDED, resultCode:
> 0, result: null
> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Publish async job-223 complete on message bus
> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Wake up jobs related to job-223
> 2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Update db status for job-223
> 2019-08-28 13:23:14,892 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
> (logid:3e502c4b) Wake up jobs joined with job-223 and disjoin all subjobs
> created from job- 223
> 2019-08-28 13:23:14,895 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
> with run of VM work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
> 2019-08-28 13:23:14,895 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
> executing com.cloud.vm.VmWorkStop for job-222
> 2019-08-28 13:23:14,898 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Remove
> job-222 from job monitoring
> 2019-08-28 13:23:14,901 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
> with run of VM work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
> 2019-08-28 13:23:14,901 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
> executing com.cloud.vm.VmWorkStop for job-223
> 2019-08-28 13:23:14,903 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Remove
> job-223 from job monitoring
> 2019-08-28 13:23:14,916 DEBUG [c.c.c.CapacityManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) VM state transitted from
> :Stopped to Expunging with event: ExpungeOperationvm's original host id:
> null new host id: null host id before state transition: null
> 2019-08-28 13:23:14,921 DEBUG [c.c.c.CapacityManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) VM state transitted from
> :Stopped to Expunging with event: ExpungeOperationvm's original host id:
> null new host id: null host id before state transition: null
> 2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Destroying vm
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up NICS
> 2019-08-28 13:23:14,923 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning network for vm: 67
> 2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Destroying vm
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up NICS
> 2019-08-28 13:23:14,929 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning network for vm: 68
> 2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking VirtualRouter to
> release Nic[184-67-null-137.74.82.208]
> 2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking SecurityGroupProvider
> to release Nic[184-67-null-137.74.82.208]
> 2019-08-28 13:23:14,936 DEBUG [c.c.n.g.DirectNetworkGuru]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Deallocate network: networkId:
> 203, ip: 137.74.82.208
> 2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking VirtualRouter to
> release Nic[187-68-null-137.74.82.209]
> 2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking SecurityGroupProvider
> to release Nic[187-68-null-137.74.82.209]
> 2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Deallocate network:
> networkId: 203, ip: 137.74.82.209
> 2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) remove nic 184 secondary ip
> 2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=184
> 2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
> entry ...
> 2019-08-28 13:23:14,943 DEBUG [c.c.n.g.DirectNetworkGuru]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) remove nic 187 secondary ip
> 2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=187
> 2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
> entry ...
> 2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=185
> 2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
> entry ...
> 2019-08-28 13:23:14,949 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=188
> 2019-08-28 13:23:14,950 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
> entry ...
> 2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=186
> 2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
> entry ...
> 2019-08-28 13:23:14,952 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up hypervisor data
> structures (ex. SRs in XenServer) for managed storage
> 2019-08-28 13:23:14,952 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning storage for vm: 67
> 2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=189
> 2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
> entry ...
> 2019-08-28 13:23:14,953 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up hypervisor data
> structures (ex. SRs in XenServer) for managed storage
> 2019-08-28 13:23:14,954 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning storage for vm: 68
> 2019-08-28 13:23:14,964 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) getCommandHostDelegation:
> class org.apache.cloudstack.storage.command.DeleteCommand
> 2019-08-28 13:23:14,964 DEBUG [c.c.h.XenServerGuru]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) We are returning the default
> host to execute commands because the command is not of Copy type.
> 2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-59a5da7a)
> (logid:2183e74c) Seq 1-5778962746846478725: Sending { Cmd , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
>
> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
> }
> 2019-08-28 13:23:14,965 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) getCommandHostDelegation:
> class org.apache.cloudstack.storage.command.DeleteCommand
> 2019-08-28 13:23:14,965 DEBUG [c.c.h.XenServerGuru]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) We are returning the default
> host to execute commands because the command is not of Copy type.
> 2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
> Sending { Cmd , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com), Ver:
> v1, Flags: 100011,
>
> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
> }
> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-2:null) (logid:) Seq 1-5778962746846478726:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-1:null) (logid:) Seq 1-5778962746846478725:
> Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
> Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
> Ver: v1, Flags: 10, { Answer } }
> 2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-59a5da7a)
> (logid:2183e74c) Seq 1-5778962746846478725: Received: { Ans: , MgmtId:
> 205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, { Answer
> }
> }
> 2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Volume 63 is not referred
> anywhere, remove it from volumes table
> 2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Volume 62 is not referred
> anywhere, remove it from volumes table
> 2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expunged
> VM[ConsoleProxy|v-68-VM]
> 2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Expunged
> VM[SecondaryStorageVm|s-67-VM]
> 2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) received secondary storage vm
> alert
> 2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Secondary Storage Vm creation
> failure, zone: France
> 2019-08-28 13:23:14,975 WARN [c.c.a.AlertManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) AlertType:: 19 |
> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
> Storage Vm creation failure. zone: France, error details: null
> 2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) received console proxy alert
> 2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Console proxy creation
> failure, zone: France
> 2019-08-28 13:23:14,976 WARN [c.c.a.AlertManagerImpl]
> (consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) AlertType:: 10 |
> dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
> proxy creation failure. zone: France, error details: null
> 2019-08-28 13:23:14,977 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
> (secstorage-1:ctx-59a5da7a) (logid:2183e74c) Primary secondary storage is
> not even started, wait until next turn
> 2019-08-28 13:23:16,259 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-bd2c1e74) (logid:dc193f19) ===START=== 137.74.35.102
> -- GET command=listInfrastructure&response=json&_=1566988128979
> 2019-08-28 13:23:16,268 DEBUG [c.c.a.ApiServer]
> (qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) CIDRs from
> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2019-08-28 13:23:16,286 DEBUG [c.c.a.ApiServlet]
> (qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) ===END===
> 137.74.35.102 -- GET
> command=listInfrastructure&response=json&_=1566988128979
> 2019-08-28 13:23:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) Begin cleanup
> expired async-jobs
> 2019-08-28 13:23:24,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) End cleanup expired
> async-jobs
> 2019-08-28 13:23:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
> 2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-3:null) (logid:) Process host VM state report from
> ping process. host: 1
> 2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
> number of records in report: 0
> 2019-08-28 13:23:24,958 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-3:null) (logid:) Done with process of VM state
> report. host: 1
>
RE: Unable to start System VMs
Posted by Paul Angus <pa...@shapeblue.com>.
The error is here Fariborz,
2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
you're already using a VLAN that you've also asked CloudStack to use.
paul.angus@shapeblue.comÂ
www.shapeblue.com
Amadeus House, Floral Street, London WC2E 9DPUK
@shapeblue
-----Original Message-----
From: Fariborz Navidan <md...@gmail.com>
Sent: 28 August 2019 12:29
To: users@cloudstack.apache.org
Subject: Unable to start System VMs
Hello,
New ACS management is unable to start System VMs. Below are logs. ACS is
still poor in error handling and reporting. It should be able to retyrn
clear error messages inside UI.
2019-08-28 13:21:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) Begin cleanup
expired async-jobs
2019-08-28 13:21:54,844 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0663ec9c) (logid:46769467) End cleanup expired
async-jobs
2019-08-28 13:22:04,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) Begin cleanup
expired async-jobs
2019-08-28 13:22:04,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-da64de3b) (logid:13fbf230) End cleanup expired
async-jobs
2019-08-28 13:22:09,812 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-42c40f78) (logid:26b61170) ===START===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566988128964
2019-08-28 13:22:09,820 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:09,839 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-42c40f78 ctx-f3a5c61e) (logid:26b61170) ===END===
137.74.35.102 -- GET
command=listZones&page=1&pagesize=20&response=json&_=1566988128964
2019-08-28 13:22:12,086 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-11b41222) (logid:34e5eff5) ===START=== 137.74.35.102
-- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
2019-08-28 13:22:12,088 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:12,092 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-11b41222 ctx-3aec20c6) (logid:34e5eff5) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128965
2019-08-28 13:22:12,222 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-dcd82c6d) (logid:181e3933) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
2019-08-28 13:22:12,223 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:12,225 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-dcd82c6d ctx-053bddc4) (logid:181e3933) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128966
2019-08-28 13:22:12,351 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a671b130) (logid:6972e719) ===START=== 137.74.35.102
-- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128967
2019-08-28 13:22:12,352 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:12,353 INFO [c.c.a.ApiServer]
(qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) Api Discovery
plugin was unable to find an api by that name or process any apis
2019-08-28 13:22:12,354 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a671b130 ctx-32f83603) (logid:6972e719) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566988128967
2019-08-28 13:22:14,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) Begin cleanup
expired async-jobs
2019-08-28 13:22:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 routers to
update status.
2019-08-28 13:22:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 VPC networks
to update Redundant State.
2019-08-28 13:22:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-99c22813) (logid:97ed8f60) Found 0 networks to
update RvR status.
2019-08-28 13:22:14,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cd7ab4f4) (logid:83e44417) End cleanup expired
async-jobs
2019-08-28 13:22:15,564 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-95484c01) (logid:1697b0f8) ===START===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
2019-08-28 13:22:15,572 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:15,592 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-95484c01 ctx-3b6e52cd) (logid:1697b0f8) ===END===
137.74.35.102 -- GET
command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566988128968
2019-08-28 13:22:18,768 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-24de6e9c) (logid:dd464d41) ===START=== 137.74.35.102
-- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
2019-08-28 13:22:18,776 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:18,794 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-24de6e9c ctx-b0d52d8f) (logid:dd464d41) ===END===
137.74.35.102 -- GET
command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128969
2019-08-28 13:22:18,969 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-02de057f) (logid:7df7fd05) ===START===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
2019-08-28 13:22:18,976 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:18,987 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-02de057f ctx-a72acff2) (logid:7df7fd05) ===END===
137.74.35.102 -- GET
command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566988128970
2019-08-28 13:22:19,115 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-d124351a) (logid:ea051db5) ===START=== 137.74.35.102
-- GET command=listApis&response=json&name=listVmwareDcs&_=1566988128971
2019-08-28 13:22:19,121 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:19,129 INFO [c.c.a.ApiServer]
(qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) Api Discovery
plugin was unable to find an api by that name or process any apis
2019-08-28 13:22:19,129 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-d124351a ctx-a706dba7) (logid:ea051db5) ===END===
137.74.35.102 -- GET
command=listApis&response=json&name=listVmwareDcs&_=1566988128971
2019-08-28 13:22:19,910 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Resetting hosts suitable for reconnect
2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Completed resetting hosts suitable for
reconnect
2019-08-28 13:22:19,912 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters already
owned by this management server
2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for clusters
already owned by this management server
2019-08-28 13:22:19,913 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Acquiring hosts for clusters not owned
by any management server
2019-08-28 13:22:19,914 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-5027d656) (logid:48df0324) Completed acquiring hosts for clusters
not owned by any management server
2019-08-28 13:22:22,777 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-8153bc20) (logid:32dfe4ef) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128972
2019-08-28 13:22:22,785 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:22,804 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-8153bc20 ctx-16497adb) (logid:32dfe4ef) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128972
2019-08-28 13:22:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) Begin cleanup
expired async-jobs
2019-08-28 13:22:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-9a1a58c9) (logid:9795c691) End cleanup expired
async-jobs
2019-08-28 13:22:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
number of records in report: 0
2019-08-28 13:22:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Done with process of VM state
report. host: 1
2019-08-28 13:22:29,995 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-b6613c33) (logid:8dc05d71) AutoScaling Monitor is
running...
2019-08-28 13:22:30,614 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-ee9f429f) (logid:2467c8a0) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128973
2019-08-28 13:22:30,622 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:30,641 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-ee9f429f ctx-474af31a) (logid:2467c8a0) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128973
2019-08-28 13:22:33,168 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) HostStatsCollector is
running...
2019-08-28 13:22:33,225 DEBUG [c.c.a.t.Request]
(StatsCollector-5:ctx-8ca5de0e) (logid:1e266084) Seq 1-5778962746846478713:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2019-08-28 13:22:33,665 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) StorageCollector is
running...
2019-08-28 13:22:33,676 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) There is no secondary
storage VM for secondary storage host Secondary1
2019-08-28 13:22:33,680 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2019-08-28 13:22:33,680 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) We are returning the
default host to execute commands because the command is not of Copy type.
2019-08-28 13:22:33,734 DEBUG [c.c.a.t.Request]
(StatsCollector-3:ctx-07390103) (logid:8f414a20) Seq 1-5778962746846478714:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2019-08-28 13:22:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) Begin cleanup
expired async-jobs
2019-08-28 13:22:34,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dd9dc584) (logid:435aef2a) End cleanup expired
async-jobs
2019-08-28 13:22:40,559 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Zone 1 is ready to launch
secondary storage VM
2019-08-28 13:22:40,562 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No running secondary storage
vms found in datacenter id=1, starting one
2019-08-28 13:22:40,567 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) No stopped secondary storage
vm is available, need to allocate a new secondary storage vm
2019-08-28 13:22:40,569 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Assign secondary storage vm
from a newly started instance for request from data center : 1
2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
configuration for offering [Network Offering
[6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
2019-08-28 13:22:40,584 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
configuration for offering [Network Offering
[3-Control-System-Control-Network]: Ntwk[201|Control|3]
2019-08-28 13:22:40,591 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Found existing network
configuration for offering [Network Offering
[2-Management-System-Management-Network]: Ntwk[200|Management|2]
2019-08-28 13:22:40,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Zone 1 is ready to launch
console proxy
2019-08-28 13:22:40,614 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Expand console proxy standby
capacity for zone France
2019-08-28 13:22:40,615 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) No stopped console proxy is
available, need to allocate a new console proxy
2019-08-28 13:22:40,616 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Assign console proxy from a
newly started instance for request from data center : 1
2019-08-28 13:22:40,623 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating entries for VM:
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,625 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nics for
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,628 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
VM[SecondaryStorageVm|s-67-VM] in network Ntwk[203|Guest|6] with requested
profile NicProfile[0-0-null-null-null
2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
configuration for offering [Network Offering
[6-Guest-DefaultSharedNetworkOfferingWithSGService]: Ntwk[203|Guest|6]
2019-08-28 13:22:40,638 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
configuration for offering [Network Offering
[3-Control-System-Control-Network]: Ntwk[201|Control|3]
2019-08-28 13:22:40,648 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Found existing network
configuration for offering [Network Offering
[2-Management-System-Management-Network]: Ntwk[200|Management|2]
2019-08-28 13:22:40,655 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Releasing lock for
Acct[193f925a-c974-11e9-b808-d05099d38681-system]
2019-08-28 13:22:40,660 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
VM[SecondaryStorageVm|s-67-VM] in network Ntwk[201|Control|3] with
requested profile null
2019-08-28 13:22:40,668 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating nic for vm
VM[SecondaryStorageVm|s-67-VM] in network Ntwk[200|Management|2] with
requested profile null
2019-08-28 13:22:40,673 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating entries for VM:
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,675 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nics for
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,676 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
VM[ConsoleProxy|v-68-VM] in network Ntwk[203|Guest|6] with requested
profile NicProfile[0-0-null-null-null
2019-08-28 13:22:40,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocating disks for
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,681 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Allocation completed for VM:
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
alert
2019-08-28 13:22:40,684 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) New secondary storage vm
created, zone: France, secStorageVm: s-67-VM, public IP: null, private IP:
null
2019-08-28 13:22:40,688 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
VM[ConsoleProxy|v-68-VM] in network Ntwk[201|Control|3] with requested
profile null
2019-08-28 13:22:40,690 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Sync job-218 execution on
object VmWorkJobQueue.67
2019-08-28 13:22:40,691 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating nic for vm
VM[ConsoleProxy|v-68-VM] in network Ntwk[200|Management|2] with requested
profile null
2019-08-28 13:22:40,694 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocating disks for
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,696 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Allocation completed for VM:
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy alert
2019-08-28 13:22:40,697 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) New console proxy created,
zone: France, proxy: v-68-VM, public IP: null, private IP: null
2019-08-28 13:22:40,701 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Sync job-219 execution on
object VmWorkJobQueue.68
2019-08-28 13:22:40,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
item: SyncQueueItemVO {id:194, queueId: 191, contentType: AsyncJob,
contentId: 218, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
13:22:40 CEST 2019}
2019-08-28 13:22:40,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
job-218
2019-08-28 13:22:40,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Execute sync-queue
item: SyncQueueItemVO {id:195, queueId: 192, contentType: AsyncJob,
contentId: 219, lastProcessMsid: 205425461278937, lastprocessNumber: 1,
lastProcessTime: Wed Aug 28 13:22:40 CEST 2019, created: Wed Aug 28
13:22:40 CEST 2019}
2019-08-28 13:22:40,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ad20db27) (logid:1632d779) Schedule queued
job-219
2019-08-28 13:22:40,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:d5030cfc) Add
job-218 into job monitoring
2019-08-28 13:22:40,872 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:8371f7e8) Add
job-219 into job monitoring
2019-08-28 13:22:40,877 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
Executing AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}
2019-08-28 13:22:40,877 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Run VM
work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:22:40,878 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
Executing AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}
2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Run VM
work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:22:40,878 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:22:40,880 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:22:40,880 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Successfully transitioned to start state for
VM[SecondaryStorageVm|s-67-VM] reservation id =
c608b88c-7d06-411e-bb84-f2d5e2b3390d
2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,881 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:22:40,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Successfully transitioned to start state for
VM[ConsoleProxy|v-68-VM] reservation id =
6c836a9c-0d16-42c3-86c7-446febb1a189
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:40,882 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,882 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching all possible resources under this Zone: 1
2019-08-28 13:22:40,882 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:40,883 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching all possible resources under this Zone: 1
2019-08-28 13:22:40,883 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:40,886 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:40,886 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:40,887 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:40,887 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:40,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:40,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 536870912 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
Requested RAM: 536870912
2019-08-28 13:22:40,889 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
available
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding to
list: 1
2019-08-28 13:22:40,890 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
suitable hosts
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) We need to allocate new storagepool for this volume
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Calling StoragePoolAllocators to find suitable pools
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:22:40,890 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 1073741824 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:22:40,890 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) System VMs will use shared storage for zone id=1
2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) LocalStoragePoolAllocator trying to find storage pool to
fit the vm
2019-08-28 13:22:40,890 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) ClusterScopeStoragePoolAllocator looking for storage pool
2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
pools will be ignored.
2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Found pools matching tags: [Pool[1|SharedMountPoint]]
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
Requested RAM: 1073741824
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
available
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:22:40,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding to
list: 1
2019-08-28 13:22:40,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
suitable hosts
2019-08-28 13:22:40,891 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking if storage pool is suitable, name: null ,poolId: 1
2019-08-28 13:22:40,891 INFO [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Storage pool null (1) does not supply IOPS capacity,
assuming enough capacity
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) We need to allocate new storagepool for this volume
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Calling StoragePoolAllocators to find suitable pools
2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking pool 1 for storage, totalSize: 984895062016,
usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold: 0.85
2019-08-28 13:22:40,892 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Destination pool id: 1
2019-08-28 13:22:40,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) System VMs will use shared storage for zone id=1
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) LocalStoragePoolAllocator trying to find storage pool to
fit the vm
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) ClusterScopeStoragePoolAllocator looking for storage pool
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Looking for pools in dc: 1 pod:1 cluster:1. Disabled
pools will be ignored.
2019-08-28 13:22:40,892 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Found pools matching tags: [Pool[1|SharedMountPoint]]
2019-08-28 13:22:40,893 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking if storage pool is suitable, name: null ,poolId: 1
2019-08-28 13:22:40,893 INFO [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Storage pool null (1) does not supply IOPS capacity,
assuming enough capacity
2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking pool 1 for storage, totalSize: 984895062016,
usedBytes: 727498752, usedPct: 7.386561066829488E-4, disable threshold: 0.85
2019-08-28 13:22:40,893 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Destination pool id: 1
2019-08-28 13:22:40,896 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Pool ID for the volume with ID 62 is null
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Found storage pool Primary1 of type SharedMountPoint
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Total capacity of the pool Primary1 with ID 1 is
984895062016
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking pool: 1 for storage allocation , maxSize :
984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
disable threshold: 0.85
2019-08-28 13:22:40,897 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) ClusterScopeStoragePoolAllocator returning 1 suitable
storage pools
2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:22:40,897 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Host: 1 can access pool: 1
2019-08-28 13:22:40,897 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Pool ID for the volume with ID 63 is null
2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Found storage pool Primary1 of type SharedMountPoint
2019-08-28 13:22:40,898 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Total capacity of the pool Primary1 with ID 1 is
984895062016
2019-08-28 13:22:40,898 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:22:40,899 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking pool: 1 for storage allocation , maxSize :
984895062016, totalAllocatedSize : 197120, askingSize : 0, allocated
disable threshold: 0.85
2019-08-28 13:22:40,899 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) ClusterScopeStoragePoolAllocator returning 1 suitable
storage pools
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(62|ROOT-->Pool(1))]
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Host: 1 can access pool: 1
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:22:40,899 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
2019-08-28 13:22:40,899 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(63|ROOT-->Pool(1))]
2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:22:40,904 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested RAM:
536870912
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
used:500, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:22:40,905 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
requested mem: 536870912,alloc_from_last:false
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU: 500
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
,Requested RAM: 1073741824
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:22:40,906 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
reserved: 0; requested mem: 1073741824,alloc_from_last:false
2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Network id=203 is already implemented
2019-08-28 13:22:40,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Network id=203 is already implemented
2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Asking VirtualRouter to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Asking VirtualRouter to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:22:40,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Asking SecurityGroupProvider to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:22:40,913 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Asking SecurityGroupProvider to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:22:40,914 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Network id=201 is already implemented
2019-08-28 13:22:40,915 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Network id=201 is already implemented
2019-08-28 13:22:40,919 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Network id=200 is already implemented
2019-08-28 13:22:40,920 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Network id=200 is already implemented
2019-08-28 13:22:40,922 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Releasing ip address for instance=186
2019-08-28 13:22:40,923 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Releasing ip address for instance=189
2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Allocated a nic
NicProfile[186-67-c608b88c-7d06-411e-bb84-f2d5e2b3390d-192.168.0.226-null
for VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:40,924 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Allocated a nic
NicProfile[189-68-6c836a9c-0d16-42c3-86c7-446febb1a189-192.168.0.121-null
for VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) template 3 is already in store:1, type:Image
2019-08-28 13:22:40,928 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) template 3 is already in store:1, type:Image
2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) template 3 is already in store:1, type:Primary
2019-08-28 13:22:40,929 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) template 3 is already in store:1, type:Primary
2019-08-28 13:22:40,932 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) copyAsync inspecting src type TEMPLATE copyAsync
inspecting dest type VOLUME
2019-08-28 13:22:40,933 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) copyAsync inspecting src type TEMPLATE copyAsync
inspecting dest type VOLUME
2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) getCommandHostDelegation: class
org.apache.cloudstack.storage.command.CopyCommand
2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) We are returning the default host to execute commands
because the target hypervisor of the source data is not XenServer.
2019-08-28 13:22:40,934 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) getCommandHostDelegation: class
org.apache.cloudstack.storage.command.CopyCommand
2019-08-28 13:22:40,934 DEBUG [c.c.h.XenServerGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) We are returning the default host to execute commands
because the target hypervisor of the source data is not XenServer.
2019-08-28 13:22:40,934 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478715: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
Template
(KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
}
2019-08-28 13:22:40,935 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478716: Waiting for Seq
5778962746846478715 Scheduling: { Cmd , MgmtId: 205425461278937, via: 1(
fr-kvm1.r9host.com), Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d1df186c-3ce8-4788-86b5-f4dd92d22304","origUrl":"
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2","uuid":"4d7c0588-85b5-455b-be42-1a0a4fed9288","id":3,"format":"QCOW2","accountId":1,"checksum":"6d12cc764cd7d64112d8c35d70923eb1","hvm":false,"displayText":"SystemVM
Template
(KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"routing-3","hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
}
2019-08-28 13:22:41,037 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-138d8f65) (logid:85352d35) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128974
2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478715:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
}
2019-08-28 13:22:41,037 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-7:null) (logid:) Seq 1-5778962746846478716: Sending
now. is current sequence.
2019-08-28 13:22:41,037 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478715: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
CopyCmdAnswer } }
2019-08-28 13:22:41,040 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:41,047 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-138d8f65 ctx-ea6cf5cf) (logid:85352d35) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128974
2019-08-28 13:22:41,057 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
2019-08-28 13:22:41,071 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478717: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
}
2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","accountId":0,"format":"QCOW2","id":0}},"result":true,"wait":0}}]
}
2019-08-28 13:22:41,126 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-6:null) (logid:) Seq 1-5778962746846478716: No more
commands found
2019-08-28 13:22:41,126 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478716: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 110, {
CopyCmdAnswer } }
2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-9:null) (logid:) Seq 1-5778962746846478717:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=192.168.0.226
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.226","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:12:00:00:ee","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:22:41,127 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478717: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:22:41,128 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:22:41,129 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Cleaning up resources for the vm
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:22:41,131 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478718: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.64","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:22:41,139 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.2.191
eth0mask=255.255.0.0 eth1ip=192.168.0.121 eth1mask=255.255.255.0 mgmtcidr=
51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
2019-08-28 13:22:41,151 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478719: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.191","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
}
2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-8:null) (logid:) Seq 1-5778962746846478719:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.2.191 eth0mask=255.255.0.0 eth1ip=192.168.0.121
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.2.191","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:bf","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.121","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:cf:00:00:85","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:22:41,319 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478719: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:22:41,323 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:22:41,326 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
in Starting state
2019-08-28 13:22:41,337 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478720: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.2.191","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:22:44,313 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-10:null) (logid:) Seq 1-5778962746846478718:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:22:44,314 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Seq 1-5778962746846478718: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:22:44,318 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
2019-08-28 13:22:44,321 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Releasing ip address for
reservationId=c608b88c-7d06-411e-bb84-f2d5e2b3390d, instance=186
2019-08-28 13:22:44,321 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Successfully released network resources for the vm
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:22:44,322 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Successfully cleaned up resources for the VM
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:22:44,323 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
2019-08-28 13:22:44,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:22:44,324 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:22:44,324 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:44,327 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:44,328 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:22:44,330 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
suitable hosts
2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) No suitable hosts found
2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) No suitable hosts found under this Cluster: 1
2019-08-28 13:22:44,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,330 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) The specified cluster is in avoid set, returning.
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:44,331 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:22:44,331 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Searching all possible resources under this Zone: 1
2019-08-28 13:22:44,332 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:44,333 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:22:44,334 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:22:44,338 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:22:44,340 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:22:44,343 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,343 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218 ctx-356d7100)
(logid:e3068c13) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,343 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:22:44,344 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Unable
to complete AsyncJobVO {id:218, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}, job origin:76
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,344 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
Complete async job-218, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13)
Publish async job-218 complete on message bus
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
up jobs related to job-218
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Update
db status for job-218
2019-08-28 13:22:44,345 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Wake
up jobs joined with job-218 and disjoin all subjobs created from job- 218
2019-08-28 13:22:44,349 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Done
executing com.cloud.vm.VmWorkStart for job-218
2019-08-28 13:22:44,350 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-136:ctx-15dac4e2 job-76/job-218) (logid:e3068c13) Remove
job-218 from job monitoring
2019-08-28 13:22:44,351 WARN [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Exception while trying to
start secondary storage vm
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,352 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Unable to start secondary
storage vm for standby capacity, vm id : 67, will recycle it and start a
new one
2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) received secondary storage vm
alert
2019-08-28 13:22:44,352 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Secondary Storage Vm creation
failure, zone: France
2019-08-28 13:22:44,352 WARN [c.c.a.AlertManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) AlertType:: 19 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
Storage Vm creation failure. zone: France, error details: null
2019-08-28 13:22:44,353 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-3a6545e7) (logid:5fd568bf) Primary secondary storage is
not even started, wait until next turn
2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-11:null) (logid:) Seq 1-5778962746846478720:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:22:44,433 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Seq 1-5778962746846478720: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:22:44,436 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
2019-08-28 13:22:44,438 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Releasing ip address for
reservationId=6c836a9c-0d16-42c3-86c7-446febb1a189, instance=189
2019-08-28 13:22:44,438 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Successfully released network resources for the vm
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:22:44,439 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Successfully cleaned up resources for the VM
VM[ConsoleProxy|v-68-VM] in Starting state
2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:22:44,440 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:22:44,440 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:22:44,441 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:22:44,443 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:22:44,444 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
suitable hosts
2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) No suitable hosts found
2019-08-28 13:22:44,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) No suitable hosts found under this Cluster: 1
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:22:44,445 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) The specified cluster is in avoid set, returning.
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:22:44,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Searching all possible resources under this Zone: 1
2019-08-28 13:22:44,446 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Zone: 1
2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:22:44,447 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:22:44,450 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:22:44,452 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,453 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219 ctx-4a0482cb)
(logid:3e502c4b) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:22:44,453 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:22:44,453 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Unable
to complete AsyncJobVO {id:219, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:22:40 CEST 2019, removed:
null}, job origin:77
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
Complete async job-219, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b)
Publish async job-219 complete on message bus
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
up jobs related to job-219
2019-08-28 13:22:44,454 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Update
db status for job-219
2019-08-28 13:22:44,455 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Wake
up jobs joined with job-219 and disjoin all subjobs created from job- 219
2019-08-28 13:22:44,457 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Done
executing com.cloud.vm.VmWorkStart for job-219
2019-08-28 13:22:44,458 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-137:ctx-41c69a85 job-77/job-219) (logid:3e502c4b) Remove
job-219 from job monitoring
2019-08-28 13:22:44,459 WARN [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Exception while trying to
start console proxy
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:22:44,460 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Unable to start console
proxy vm for standby capacity, vm id : 68, will recycle it and start a new
one
2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) received console proxy alert
2019-08-28 13:22:44,460 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) Console proxy creation
failure, zone: France
2019-08-28 13:22:44,460 WARN [c.c.a.AlertManagerImpl]
(consoleproxy-1:ctx-71d4f124) (logid:84bfc356) AlertType:: 10 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
proxy creation failure. zone: France, error details: null
2019-08-28 13:22:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) Begin cleanup
expired async-jobs
2019-08-28 13:22:44,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 routers to
update status.
2019-08-28 13:22:44,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 VPC networks
to update Redundant State.
2019-08-28 13:22:44,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bede0d96) (logid:d44dfa1d) Found 0 networks to
update RvR status.
2019-08-28 13:22:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-681c867e) (logid:bc191977) End cleanup expired
async-jobs
2019-08-28 13:22:48,876 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-1ca6b8a4) (logid:767cda38) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128975
2019-08-28 13:22:48,883 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:48,903 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-1ca6b8a4 ctx-37a1ca82) (logid:767cda38) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128975
2019-08-28 13:22:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) Begin cleanup
expired async-jobs
2019-08-28 13:22:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-dabb31ef) (logid:77d4ad43) End cleanup expired
async-jobs
2019-08-28 13:22:56,741 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-3fa512ab) (logid:4609e172) ===START===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128976
2019-08-28 13:22:56,747 DEBUG [c.c.a.ApiServer]
(qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:22:56,751 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-267:ctx-3fa512ab ctx-4ae287f9) (logid:4609e172) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128976
2019-08-28 13:23:03,063 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a0c69e67) (logid:0a2b6c35) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128977
2019-08-28 13:23:03,071 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:23:03,091 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-a0c69e67 ctx-5787b918) (logid:0a2b6c35) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128977
2019-08-28 13:23:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) Begin cleanup
expired async-jobs
2019-08-28 13:23:04,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cb690e8b) (logid:dc112e6a) End cleanup expired
async-jobs
2019-08-28 13:23:09,900 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-16:ctx-9f3738f2) (logid:dea95b1b) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128978
2019-08-28 13:23:09,907 DEBUG [c.c.a.ApiServer]
(qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:23:09,929 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-16:ctx-9f3738f2 ctx-215a1fff) (logid:dea95b1b) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128978
2019-08-28 13:23:10,557 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Zone 1 is ready to launch
secondary storage VM
2019-08-28 13:23:10,561 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) No running secondary storage
vms found in datacenter id=1, starting one
2019-08-28 13:23:10,570 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Found a stopped secondary
storage vm, starting it. Vm id : 67
2019-08-28 13:23:10,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-220 execution on
object VmWorkJobQueue.67
2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Zone 1 is ready to launch
console proxy
2019-08-28 13:23:10,613 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expand console proxy standby
capacity for zone France
2019-08-28 13:23:10,616 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Found a stopped console
proxy, starting it. Vm id : 68
2019-08-28 13:23:10,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-221 execution on
object VmWorkJobQueue.68
2019-08-28 13:23:10,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
item: SyncQueueItemVO {id:196, queueId: 191, contentType: AsyncJob,
contentId: 220, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
13:23:10 CEST 2019}
2019-08-28 13:23:10,858 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
job-220
2019-08-28 13:23:10,861 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Execute sync-queue
item: SyncQueueItemVO {id:197, queueId: 192, contentType: AsyncJob,
contentId: 221, lastProcessMsid: 205425461278937, lastprocessNumber: 2,
lastProcessTime: Wed Aug 28 13:23:10 CEST 2019, created: Wed Aug 28
13:23:10 CEST 2019}
2019-08-28 13:23:10,864 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b6db927) (logid:ee885c57) Schedule queued
job-221
2019-08-28 13:23:10,865 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:f78794fa) Add
job-220 into job monitoring
2019-08-28 13:23:10,871 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:7deee6cf) Add
job-221 into job monitoring
2019-08-28 13:23:10,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
Executing AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}
2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Run VM
work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:23:10,880 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:10,881 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
Executing AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}
2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Run VM
work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:23:10,881 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:10,883 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:23:10,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Successfully transitioned to start state for
VM[SecondaryStorageVm|s-67-VM] reservation id =
46a0e954-a9af-45af-ac39-91e036654f95
2019-08-28 13:23:10,884 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) VM state transitted from :Stopped to Starting with event:
StartRequestedvm's original host id: null new host id: null host id before
state transition: null
2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Successfully transitioned to start state for
VM[ConsoleProxy|v-68-VM] reservation id =
a25c0118-b49c-41b2-8970-9b73191746f3
2019-08-28 13:23:10,884 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:10,885 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:10,885 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:23:10,885 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: null
2019-08-28 13:23:10,886 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:10,888 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:10,888 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:23:10,889 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host: 1 has cpu capability
(cpu:12, speed:4500) to support requested CPU: 1 and requested speed: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 1073741824 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: 536870912 ,
cpuOverprovisioningFactor: 4.0
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Free RAM: 66088833024 ,
Requested RAM: 1073741824
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host has enough CPU and RAM
available
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
1073741824,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Found a suitable host, adding to
list: 1
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 1
suitable hosts
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Hosts's actual total CPU: 54000
and CPU after applying overprovisioning: 216000
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Free CPU: 216000 , Requested
CPU: 500
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Free RAM: 66088833024 ,
Requested RAM: 536870912
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host has enough CPU and RAM
available
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc CPU from host:
1, used: 0, reserved: 0, actual total: 54000, total with overprovisioning:
216000; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) STATS: Can alloc MEM from host:
1, used: 0, reserved: 0, total: 66088833024; requested mem:
536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Found a suitable host, adding to
list: 1
2019-08-28 13:23:10,891 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 1
suitable hosts
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking suitable pools for volume (Id, Type): (63,ROOT)
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Volume has pool already allocated, checking if pool can be
reused, poolId: 1
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking suitable pools for volume (Id, Type): (62,ROOT)
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Volume has pool already allocated, checking if pool can be
reused, poolId: 1
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Planner need not allocate a pool for this volume since its
READY
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Planner need not allocate a pool for this volume since its
READY
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to find a potenial host and associated storage
pools from the suitable host/pool lists for this VM
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking if host: 1 can access any suitable storage pool
for volume: ROOT
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Host: 1 can access pool: 1
2019-08-28 13:23:10,892 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Host: 1 can access pool: 1
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Found a potential host id: 1 name: fr-kvm1.r9host.com and
associated storage pools for this VM
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deployment found - P0=VM[SecondaryStorageVm|s-67-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,893 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Returning Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,893 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deployment found - P0=VM[ConsoleProxy|v-68-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:23:10,899 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) VM state transitted from :Starting to Starting with event:
OperationRetryvm's original host id: null new host id: 1 host id before
state transition: null
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Current Used CPU: 0 , Free CPU:216000 ,Requested CPU: 500
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Current Used RAM: 0 , Free RAM:66088833024 ,Requested RAM:
536870912
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) CPU STATS after allocation: for host: 1, old used: 0, old
reserved: 0, actual total: 54000, total with overprovisioning: 216000; new
used:500, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) RAM STATS after allocation: for host: 1, old used: 0, old
reserved: 0, total: 66088833024; new used: 536870912, reserved: 0;
requested mem: 536870912,alloc_from_last:false
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) We are allocating VM, increasing the used capacity of this
host:1
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Current Used CPU: 500 , Free CPU:215500 ,Requested CPU: 500
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Current Used RAM: 536870912 , Free RAM:65551962112
,Requested RAM: 1073741824
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) CPU STATS after allocation: for host: 1, old used: 500,
old reserved: 0, actual total: 54000, total with overprovisioning: 216000;
new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
2019-08-28 13:23:10,901 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) RAM STATS after allocation: for host: 1, old used:
536870912, old reserved: 0, total: 66088833024; new used: 1610612736,
reserved: 0; requested mem: 1073741824,alloc_from_last:false
2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Network id=203 is already implemented
2019-08-28 13:23:10,904 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Network id=203 is already implemented
2019-08-28 13:23:10,907 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Asking VirtualRouter to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Asking VirtualRouter to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:10,908 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Asking SecurityGroupProvider to prepare for
Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:10,909 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Asking SecurityGroupProvider to prepare for
Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:10,911 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Network id=201 is already implemented
2019-08-28 13:23:10,912 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Network id=201 is already implemented
2019-08-28 13:23:10,916 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Network id=200 is already implemented
2019-08-28 13:23:10,918 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Network id=200 is already implemented
2019-08-28 13:23:10,918 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Releasing ip address for instance=186
2019-08-28 13:23:10,920 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Allocated a nic
NicProfile[186-67-46a0e954-a9af-45af-ac39-91e036654f95-192.168.0.86-null
for VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:10,921 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Releasing ip address for instance=189
2019-08-28 13:23:10,922 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No need to recreate the volume: Vol[62|vm=67|ROOT], since
it already has a pool assigned: 1, adding disk to VM
2019-08-28 13:23:10,923 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Allocated a nic
NicProfile[189-68-a25c0118-b49c-41b2-8970-9b73191746f3-192.168.0.20-null
for VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:10,925 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No need to recreate the volume: Vol[63|vm=68|ROOT], since
it already has a pool assigned: 1, adding disk to VM
2019-08-28 13:23:10,930 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Boot Args for VM[SecondaryStorageVm|s-67-VM]:
template=domP type=secstorage host=51.91.247.86 port=8250 name=s-67-VM
zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4 nfsVersion=null
2019-08-28 13:23:10,931 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Boot Args for VM[ConsoleProxy|v-68-VM]: template=domP
type=consoleproxy host=51.91.247.86 port=8250 name=v-68-VM zone=1 pod=1
guid=Proxy.68 proxy_vm=68 disable_rp_filter=true eth2ip=137.74.82.209
eth2mask=255.255.255.240 gateway=137.74.82.222 eth0ip=169.254.0.248
eth0mask=255.255.0.0 eth1ip=192.168.0.20 eth1mask=255.255.255.0 mgmtcidr=
51.91.247.0/24 localgw=192.168.0.2 internaldns1=8.8.8.8
internaldns2=8.8.4.4 dns1=8.8.8.8 dns2=8.8.4.4
2019-08-28 13:23:10,941 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478721: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.170","port":3922,"interval":6,"retries":100,"name":"s-67-VM","wait":0}}]
}
2019-08-28 13:23:10,942 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478722: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":false,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"hostIp":"192.168.0.2","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.248","port":3922,"interval":6,"retries":100,"name":"v-68-VM","wait":0}}]
}
2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-12:null) (logid:) Seq 1-5778962746846478722:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":68,"name":"v-68-VM","state":"Starting","type":"ConsoleProxy","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=consoleproxy host=51.91.247.86 port=8250
name=v-68-VM zone=1 pod=1 guid=Proxy.68 proxy_vm=68 disable_rp_filter=true
eth2ip=137.74.82.209 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.0.248 eth0mask=255.255.0.0 eth1ip=192.168.0.20
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
internaldns1=8.8.8.8 internaldns2=8.8.4.4 dns1=8.8.8.8
dns2=8.8.4.4","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sWwWVjZFHBmUgNGijboZWA","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"758fb0a9-1461-4483-bd22-e449d827f525","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"7ff915bc-5f72-4adc-8f0b-b8e026347230","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.209","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:46:00:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"99af9032-f3c5-4db6-9af3-00b53ab28a61","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.0.248","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:f8","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"25f58bf0-4450-40bf-a077-41b37a8e5116","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.20","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:bc:00:00:20","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:23:10,984 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478722: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:23:10,985 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:23:10,986 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Cleaning up resources for the vm VM[ConsoleProxy|v-68-VM]
in Starting state
2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478723: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.0.248","forceStop":false,"volumesToDisconnect":[],"vmName":"v-68-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-13:null) (logid:) Seq 1-5778962746846478721:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":67,"name":"s-67-VM","state":"Starting","type":"SecondaryStorageVm","cpus":1,"minSpeed":125,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
5","bootArgs":" template=domP type=secstorage host=51.91.247.86 port=8250
name=s-67-VM zone=1 pod=1 guid=s-67-VM workers=5
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=false role=templateProcessor mtu=1500
eth2ip=137.74.82.208 eth2mask=255.255.255.240 gateway=137.74.82.222
eth0ip=169.254.1.170 eth0mask=255.255.0.0 eth1ip=192.168.0.86
eth1mask=255.255.255.0 mgmtcidr=51.91.247.0/24 localgw=192.168.0.2
private.network.device=eth1 internaldns1=8.8.8.8 internaldns2=8.8.4.4
dns1=8.8.8.8 dns2=8.8.4.4
nfsVersion=null","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"deGxTO_1xS3VpN6bKpGakg","vncAddr":"192.168.0.2","params":{"cpuOvercommitRatio":"4","memoryOvercommitRatio":"1.0"},"uuid":"4c4005cd-0a42-4af3-8ae6-08e0d8714728","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"0395ebee-1622-418b-85eb-20c351034397","type":"ROOT","_details":{"storageHost":"localhost","managed":"false","storagePort":"0","volumeSize":"0"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"9b7ad00a-ec0e-47a5-a15a-8075905fc4e5","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"a3caad47-d749-40d7-bacb-fee988c8b99b","ip":"137.74.82.208","netmask":"255.255.255.240","gateway":"137.74.82.222","mac":"1e:00:61:00:00:01","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":true,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"646280b9-e15d-4a1d-b2e2-453b43aee37d","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"e3559d66-4654-426a-a1ad-5f22567da00d","ip":"169.254.1.170","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:aa","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"f407a612-43c2-4f72-9afa-89d6fbe61b4a","details":{"ForgedTransmits":"true","PromiscuousMode":"false","MacAddressChanges":"true"},"dpdkDisabled":true,"uuid":"c0c84838-a5be-4461-827f-f6f5ca919dbd","ip":"192.168.0.86","netmask":"255.255.255.0","gateway":"192.168.0.2","mac":"1e:00:d2:00:00:62","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}],"guestOsDetails":{},"extraConfig":{}},"result":false,"details":"Failed
to create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to
breth1-100","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
2019-08-28 13:23:10,988 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478721: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StartAnswer, Answer } }
2019-08-28 13:23:10,989 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Unable to start VM on Host[-1-Routing] due to Failed to
create vnet 100: device eth1.100 is already a member of a bridge; can't
enslave it to bridge breth1-100.Failed to add vlan: eth1.100 to breth1-100
2019-08-28 13:23:10,990 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Cleaning up resources for the vm
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:23:10,991 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478724: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.1.170","forceStop":false,"volumesToDisconnect":[],"vmName":"s-67-VM","executeInSequence":false,"wait":0}}]
}
2019-08-28 13:23:14,130 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-14:null) (logid:) Seq 1-5778962746846478724:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,131 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Seq 1-5778962746846478724: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:23:14,136 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Released nic: NicProfile[185-67-null-null-null
2019-08-28 13:23:14,138 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Releasing ip address for
reservationId=46a0e954-a9af-45af-ac39-91e036654f95, instance=186
2019-08-28 13:23:14,139 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Released nic: NicProfile[186-67-null-null-null
2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Successfully released network resources for the vm
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,140 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Successfully cleaned up resources for the VM
VM[SecondaryStorageVm|s-67-VM] in Starting state
2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:14,141 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Vol[62|vm=67|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:14,141 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:23:14,142 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,145 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:14,145 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:14,146 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Looking for speed=500Mhz, Ram=512
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:23:14,147 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae
FirstFitRoutingAllocator) (logid:e3068c13) Host Allocator returning 0
suitable hosts
2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No suitable hosts found
2019-08-28 13:23:14,147 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No suitable hosts found under this Cluster: 1
2019-08-28 13:23:14,148 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,148 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) The specified cluster is in avoid set, returning.
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:null, requested cpu: 500, requested ram: 536870912
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:23:14,149 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Searching resources only under specified Pod: 1
2019-08-28 13:23:14,149 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Pod: 1
2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:23:14,151 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:23:14,156 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) release cpu from host: 1, old used: 1000,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:23:14,158 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) release mem from host: 1, old used: 1610612736,reserved:
0, total: 66088833024; new used: 1073741824,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:23:14,160 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,161 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220 ctx-bb1ddfae)
(logid:e3068c13) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,161 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 67, job origin: 76
2019-08-28 13:23:14,161 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Unable
to complete AsyncJobVO {id:220, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}, job origin:76
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
Complete async job-220, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTY3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13)
Publish async job-220 complete on message bus
2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
up jobs related to job-220
2019-08-28 13:23:14,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Update
db status for job-220
2019-08-28 13:23:14,163 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Wake
up jobs joined with job-220 and disjoin all subjobs created from job- 220
2019-08-28 13:23:14,166 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Done
executing com.cloud.vm.VmWorkStart for job-220
2019-08-28 13:23:14,166 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-138:ctx-c18389dc job-76/job-220) (logid:e3068c13) Remove
job-220 from job monitoring
2019-08-28 13:23:14,169 WARN [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Exception while trying to
start secondary storage vm
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[SecondaryStorageVm|s-67-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,169 INFO [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Unable to start secondary
storage vm for standby capacity, vm id : 67, will recycle it and start a
new one
2019-08-28 13:23:14,172 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Sync job-222 execution on
object VmWorkJobQueue.67
2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-15:null) (logid:) Seq 1-5778962746846478723:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,618 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Seq 1-5778962746846478723: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, {
StopAnswer } }
2019-08-28 13:23:14,623 DEBUG [c.c.n.g.ControlNetworkGuru]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Released nic: NicProfile[188-68-null-null-null
2019-08-28 13:23:14,626 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Releasing ip address for
reservationId=a25c0118-b49c-41b2-8970-9b73191746f3, instance=189
2019-08-28 13:23:14,626 DEBUG [c.c.n.g.PodBasedNetworkGuru]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Released nic: NicProfile[189-68-null-null-null
2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Successfully released network resources for the vm
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,628 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Successfully cleaned up resources for the VM
VM[ConsoleProxy|v-68-VM] in Starting state
2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Root volume is ready, need to place VM in volume's cluster
2019-08-28 13:23:14,629 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Vol[63|vm=68|ROOT] is READY, changing deployment plan to
use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: Yes
2019-08-28 13:23:14,630 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: null, hosts: [1]
2019-08-28 13:23:14,631 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,635 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Checking resources in Cluster: 1 under Pod: 1
2019-08-28 13:23:14,635 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for hosts in dc: 1
pod:1 cluster:1
2019-08-28 13:23:14,636 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) FirstFitAllocator has 1 hosts to
check for allocation: [Host[-1-Routing]]
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Found 1 hosts for allocation
after prioritization: [Host[-1-Routing]]
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Looking for speed=500Mhz,
Ram=1024
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host name: fr-kvm1.r9host.com,
hostId: 1 is in avoid set, skipping this and trying other available hosts
2019-08-28 13:23:14,638 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385
FirstFitRoutingAllocator) (logid:3e502c4b) Host Allocator returning 0
suitable hosts
2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No suitable hosts found
2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No suitable hosts found under this Cluster: 1
2019-08-28 13:23:14,638 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Could not find suitable Deployment Destination for this VM
under any clusters, returning.
2019-08-28 13:23:14,638 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Cluster: 1
2019-08-28 13:23:14,639 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) The specified cluster is in avoid set, returning.
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) DeploymentPlanner allocation algorithm: null
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:null, requested cpu: 500, requested ram: 1073741824
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Is ROOT volume READY (pool already allocated)?: No
2019-08-28 13:23:14,639 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Deploy avoids pods: null, clusters: [1], hosts: [1]
2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Searching resources only under specified Pod: 1
2019-08-28 13:23:14,640 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Listing clusters in order of aggregate capacity, that have
(atleast one host with) enough CPU and RAM capacity under this Pod: 1
2019-08-28 13:23:14,641 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Removing from the clusterId list these clusters from avoid
set: [1]
2019-08-28 13:23:14,642 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) No clusters found after removing disabled clusters and
clusters in avoid list, returning.
2019-08-28 13:23:14,646 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: null new host id: null host id before
state transition: 1
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Hosts's actual total CPU: 54000 and CPU after applying
overprovisioning: 216000
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Hosts's actual total RAM: 66088833024 and RAM after
applying overprovisioning: 66088833024
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) release cpu from host: 1, old used: 500,reserved: 0,
actual total: 54000, total with overprovisioning: 216000; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2019-08-28 13:23:14,649 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) release mem from host: 1, old used: 1073741824,reserved:
0, total: 66088833024; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2019-08-28 13:23:14,650 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,651 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221 ctx-cb615385)
(logid:3e502c4b) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2019-08-28 13:23:14,651 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 68, job origin: 77
2019-08-28 13:23:14,651 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Unable
to complete AsyncJobVO {id:221, userId: 1, accountId: 1, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:10 CEST 2019, removed:
null}, job origin:77
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
Complete async job-221, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAA6VW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW0NvbnNvbGVQcm94eXx2LTY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAAEJ3QAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAE_txAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyMzRwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAFJ5xAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAJHdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAITcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcmNB93O_lXyzAgAAeHAA
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b)
Publish async job-221 complete on message bus
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
up jobs related to job-221
2019-08-28 13:23:14,652 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Update
db status for job-221
2019-08-28 13:23:14,653 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Wake
up jobs joined with job-221 and disjoin all subjobs created from job- 221
2019-08-28 13:23:14,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Done
executing com.cloud.vm.VmWorkStart for job-221
2019-08-28 13:23:14,656 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-139:ctx-554e1444 job-77/job-221) (logid:3e502c4b) Remove
job-221 from job monitoring
2019-08-28 13:23:14,658 WARN [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Exception while trying to
start console proxy
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[ConsoleProxy|v-68-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1063)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5115)
at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5278)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-28 13:23:14,658 INFO [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Unable to start console
proxy vm for standby capacity, vm id : 68, will recycle it and start a new
one
2019-08-28 13:23:14,662 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Sync job-223 execution on
object VmWorkJobQueue.68
2019-08-28 13:23:14,852 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 routers to
update status.
2019-08-28 13:23:14,854 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 VPC networks
to update Redundant State.
2019-08-28 13:23:14,855 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
item: SyncQueueItemVO {id:198, queueId: 191, contentType: AsyncJob,
contentId: 222, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
13:23:14 CEST 2019}
2019-08-28 13:23:14,856 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-3fd7eb60) (logid:4d9e69c3) Found 0 networks to
update RvR status.
2019-08-28 13:23:14,856 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
job-222
2019-08-28 13:23:14,859 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Execute sync-queue
item: SyncQueueItemVO {id:199, queueId: 192, contentType: AsyncJob,
contentId: 223, lastProcessMsid: 205425461278937, lastprocessNumber: 3,
lastProcessTime: Wed Aug 28 13:23:14 CEST 2019, created: Wed Aug 28
13:23:14 CEST 2019}
2019-08-28 13:23:14,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f6b81fda) (logid:e267de97) Schedule queued
job-223
2019-08-28 13:23:14,863 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:7e4a2784) Add
job-222 into job monitoring
2019-08-28 13:23:14,869 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:26933ca4) Add
job-223 into job monitoring
2019-08-28 13:23:14,871 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) Begin cleanup
expired async-jobs
2019-08-28 13:23:14,875 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13)
Executing AsyncJobVO {id:222, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
null}
2019-08-28 13:23:14,875 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Run VM
work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
2019-08-28 13:23:14,877 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,879 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-f0c803f2) (logid:f1562871) End cleanup expired
async-jobs
2019-08-28 13:23:14,880 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b)
Executing AsyncJobVO {id:223, userId: 1, accountId: 1, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAARHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 205425461278937, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Aug 28 13:23:14 CEST 2019, removed:
null}
2019-08-28 13:23:14,881 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Run VM
work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
2019-08-28 13:23:14,882 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,883 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) VM is already stopped: VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,883 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":67,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,883 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Complete async job-222, jobStatus: SUCCEEDED, resultCode:
0, result: null
2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Publish async job-222 complete on message bus
2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Wake up jobs related to job-222
2019-08-28 13:23:14,885 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Update db status for job-222
2019-08-28 13:23:14,886 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222 ctx-d7af1fea)
(logid:e3068c13) Wake up jobs joined with job-222 and disjoin all subjobs
created from job- 222
2019-08-28 13:23:14,888 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) VM is already stopped: VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,888 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":68,"handlerName":"VirtualMachineManagerImpl"}
2019-08-28 13:23:14,888 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Complete async job-223, jobStatus: SUCCEEDED, resultCode:
0, result: null
2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Publish async job-223 complete on message bus
2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Wake up jobs related to job-223
2019-08-28 13:23:14,890 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Update db status for job-223
2019-08-28 13:23:14,892 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223 ctx-8e25168c)
(logid:3e502c4b) Wake up jobs joined with job-223 and disjoin all subjobs
created from job- 223
2019-08-28 13:23:14,895 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 67, job origin: 76
2019-08-28 13:23:14,895 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Done
executing com.cloud.vm.VmWorkStop for job-222
2019-08-28 13:23:14,898 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-140:ctx-21318d3c job-76/job-222) (logid:e3068c13) Remove
job-222 from job monitoring
2019-08-28 13:23:14,901 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 68, job origin: 77
2019-08-28 13:23:14,901 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Done
executing com.cloud.vm.VmWorkStop for job-223
2019-08-28 13:23:14,903 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-141:ctx-9d74c207 job-77/job-223) (logid:3e502c4b) Remove
job-223 from job monitoring
2019-08-28 13:23:14,916 DEBUG [c.c.c.CapacityManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) VM state transitted from
:Stopped to Expunging with event: ExpungeOperationvm's original host id:
null new host id: null host id before state transition: null
2019-08-28 13:23:14,921 DEBUG [c.c.c.CapacityManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) VM state transitted from
:Stopped to Expunging with event: ExpungeOperationvm's original host id:
null new host id: null host id before state transition: null
2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Destroying vm
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,923 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up NICS
2019-08-28 13:23:14,923 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning network for vm: 67
2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Destroying vm
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,929 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up NICS
2019-08-28 13:23:14,929 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning network for vm: 68
2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking VirtualRouter to
release Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:14,936 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Asking SecurityGroupProvider
to release Nic[184-67-null-137.74.82.208]
2019-08-28 13:23:14,936 DEBUG [c.c.n.g.DirectNetworkGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Deallocate network: networkId:
203, ip: 137.74.82.208
2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking VirtualRouter to
release Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:14,940 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Asking SecurityGroupProvider
to release Nic[187-68-null-137.74.82.209]
2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Deallocate network:
networkId: 203, ip: 137.74.82.209
2019-08-28 13:23:14,940 DEBUG [c.c.n.g.DirectNetworkGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) remove nic 184 secondary ip
2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=184
2019-08-28 13:23:14,943 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,943 DEBUG [c.c.n.g.DirectNetworkGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) remove nic 187 secondary ip
2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=187
2019-08-28 13:23:14,946 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=185
2019-08-28 13:23:14,947 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,949 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=188
2019-08-28 13:23:14,950 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Removed nic id=186
2019-08-28 13:23:14,951 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,952 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning up hypervisor data
structures (ex. SRs in XenServer) for managed storage
2019-08-28 13:23:14,952 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Cleaning storage for vm: 67
2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Removed nic id=189
2019-08-28 13:23:14,953 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Revoving nic secondary ip
entry ...
2019-08-28 13:23:14,953 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning up hypervisor data
structures (ex. SRs in XenServer) for managed storage
2019-08-28 13:23:14,954 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Cleaning storage for vm: 68
2019-08-28 13:23:14,964 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) getCommandHostDelegation:
class org.apache.cloudstack.storage.command.DeleteCommand
2019-08-28 13:23:14,964 DEBUG [c.c.h.XenServerGuru]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) We are returning the default
host to execute commands because the command is not of Copy type.
2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-59a5da7a)
(logid:2183e74c) Seq 1-5778962746846478725: Sending { Cmd , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0395ebee-1622-418b-85eb-20c351034397","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-67","size":0,"path":"0395ebee-1622-418b-85eb-20c351034397","volumeId":62,"vmName":"s-67-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":62,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
}
2019-08-28 13:23:14,965 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) getCommandHostDelegation:
class org.apache.cloudstack.storage.command.DeleteCommand
2019-08-28 13:23:14,965 DEBUG [c.c.h.XenServerGuru]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) We are returning the default
host to execute commands because the command is not of Copy type.
2019-08-28 13:23:14,965 DEBUG [c.c.a.t.Request]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
Sending { Cmd , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com), Ver:
v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7b93d407-f010-4589-a90c-f835672e69e8","id":1,"poolType":"SharedMountPoint","host":"localhost","path":"/home/primary1","port":0,"url":"SharedMountPoint://localhost/home/primary1/?ROLE=Primary&STOREUUID=7b93d407-f010-4589-a90c-f835672e69e8","isManaged":false}},"name":"ROOT-68","size":0,"path":"acf69142-d239-4f8e-997b-6d0fcfca59cf","volumeId":63,"vmName":"v-68-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":63,"deviceId":0,"hypervisorType":"KVM"}},"wait":0}}]
}
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-2:null) (logid:) Seq 1-5778962746846478726:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-1:null) (logid:) Seq 1-5778962746846478725:
Processing: { Ans: , MgmtId: 205425461278937, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Seq 1-5778962746846478726:
Received: { Ans: , MgmtId: 205425461278937, via: 1(fr-kvm1.r9host.com),
Ver: v1, Flags: 10, { Answer } }
2019-08-28 13:23:14,968 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-59a5da7a)
(logid:2183e74c) Seq 1-5778962746846478725: Received: { Ans: , MgmtId:
205425461278937, via: 1(fr-kvm1.r9host.com), Ver: v1, Flags: 10, { Answer }
}
2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Volume 63 is not referred
anywhere, remove it from volumes table
2019-08-28 13:23:14,971 INFO [o.a.c.s.v.VolumeServiceImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Volume 62 is not referred
anywhere, remove it from volumes table
2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Expunged
VM[ConsoleProxy|v-68-VM]
2019-08-28 13:23:14,974 DEBUG [c.c.v.VirtualMachineManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Expunged
VM[SecondaryStorageVm|s-67-VM]
2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) received secondary storage vm
alert
2019-08-28 13:23:14,975 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Secondary Storage Vm creation
failure, zone: France
2019-08-28 13:23:14,975 WARN [c.c.a.AlertManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) AlertType:: 19 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Secondary
Storage Vm creation failure. zone: France, error details: null
2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) received console proxy alert
2019-08-28 13:23:14,976 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) Console proxy creation
failure, zone: France
2019-08-28 13:23:14,976 WARN [c.c.a.AlertManagerImpl]
(consoleproxy-1:ctx-01a17ac8) (logid:b1952d54) AlertType:: 10 |
dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Console
proxy creation failure. zone: France, error details: null
2019-08-28 13:23:14,977 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl]
(secstorage-1:ctx-59a5da7a) (logid:2183e74c) Primary secondary storage is
not even started, wait until next turn
2019-08-28 13:23:16,259 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-bd2c1e74) (logid:dc193f19) ===START=== 137.74.35.102
-- GET command=listInfrastructure&response=json&_=1566988128979
2019-08-28 13:23:16,268 DEBUG [c.c.a.ApiServer]
(qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) CIDRs from
which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2019-08-28 13:23:16,286 DEBUG [c.c.a.ApiServlet]
(qtp2114650936-14:ctx-bd2c1e74 ctx-f06c28cc) (logid:dc193f19) ===END===
137.74.35.102 -- GET
command=listInfrastructure&response=json&_=1566988128979
2019-08-28 13:23:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) Begin cleanup
expired async-jobs
2019-08-28 13:23:24,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-0637b73b) (logid:92e933a9) End cleanup expired
async-jobs
2019-08-28 13:23:24,957 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) Ping from 1(fr-kvm1.r9host.com)
2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process host VM state report from
ping process. host: 1
2019-08-28 13:23:24,957 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Process VM state report. host: 1,
number of records in report: 0
2019-08-28 13:23:24,958 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-3:null) (logid:) Done with process of VM state
report. host: 1