You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Piotr Pisz <pp...@post.pl> on 2021/11/17 13:55:11 UTC
configdrive not working in 4.16
Hi,
I have upgraded to 4.16 and am having trouble booting my VM on L2 network
with ConfigDrive. I would have a request for help.
Regards,
Piotr
2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
overprovisioning: 31200
2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) We are allocating VM, increasing the used capacity of this
host:1
2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU: 3900
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
61916983296 ,Requested RAM: (4,00 GB) 4294967296
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) We are allocating VM to the last host again, so adjusting
the reserved capacity if it is not less than required
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: (4,00
GB) 4294967296
2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0, old
reserved: 3900, actual total: 31200, total with overprovisioning: 31200; new
used:3900, reserved:0; requested cpu:3900,alloc_from_last:true
2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0; requested mem: (4,00
GB) 4294967296,alloc_from_last:true
2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to support
requested CPU: 1 and requested speed: 3900
2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Checking if host: 1 has enough capacity for requested CPU:
3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
1.0
2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
overprovisioning: 31200
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) We need to allocate to the last host again, so checking if
there is enough reserved capacity
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: (4,00
GB) 4294967296
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Host has enough CPU and RAM available
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved: 3900,
actual total: 31200, total with overprovisioning: 31200; requested
cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592; requested
mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
considerReservedCapacity?: true
2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Network id=212 is already implemented
2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Service SecurityGroup is not supported in the network
id=212
2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Changing active number of nics for network id=212 on 1
2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Asking ConfigDrive to prepare for
Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=service-offering, content=KVM-RBD-Custom
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=availability-zone, content=Piszki Lab
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=local-hostname, content=Brooklyn
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=local-ipv4, content=192.168.40.243
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=public-ipv4, content=192.168.40.243
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=public-hostname, content=null
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=public-keys, content=null
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=cloud-identifier,
content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=hypervisor-host-name, content=psc1.piszki.lab
2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Executing config drive creation command:
/usr/bin/genisoimage -o /tmp/configdrive12981467783887013707/i-6-83-VM.iso
-ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
config-2 /tmp/configdrive12981467783887013707
2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
/tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots -allow-lowercase
-allow-multidot -cache-inodes -l -quiet -J -r -V config-2
/tmp/configdrive12981467783887013707
2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Executing while with timeout : 300000
2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Execution is successful.
2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Warning: creating filesystem that does not conform to
ISO-9660.
2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/
i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{
"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupport
edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
"false"}}] }
2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfully
saved config drive at secondary
storage","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Details from executing class
com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved config
drive at secondary storage
2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Cleaning up resources for the vm VM instance {id: "83",
name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
type="User"} in Starting state
2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToD
isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","b
ypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMa
intenance":"false"}}] }
2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, { StopAnswer }
}
2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Service SecurityGroup is not supported in the network
id=212
2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Changing active number of nics for network id=212 on -1
2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Asking ConfigDrive to release
NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-null]
2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/
i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsuppor
tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance
":"false"}}] }
2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHostMainte
nance":"false"}}] }
2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Unable to release some network resources for the VM VM
instance {id: "83", name: "i-6-83-VM", uuid:
"6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
at
com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIso(Con
figDriveNetworkElement.java:661)
at
com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriveNetwo
rkElement.java:167)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.releaseNic(Ne
tworkOrchestrator.java:2238)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release(Netwo
rkOrchestrator.java:2164)
at
com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachineMana
gerImpl.java:1836)
at
com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerImpl.jav
a:1826)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:1373)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:5802)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
ccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
gMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
a:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
Impl.java:5967)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
ntextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
ultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
text(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
ext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
extRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
ManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Successfully released storage resources for the VM VM
instance {id: "83", name: "i-6-83-VM", uuid:
"6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Successfully cleaned up resources for the VM VM instance
{id: "83", name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
type="User"} in Starting state
2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
"6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited from
[Starting] to [Stopped] with event [OperationFailed]. VM's original host:
Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
"173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host: null,
host before state transition: Host {"id": "1", "name": "psc1.piszki.lab",
"uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
overprovisioning: 31200
2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and RAM
after applying overprovisioning: (61,66 GB) 66211950592
2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
actual total: 31200, total with overprovisioning: 31200; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new used:
(0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
false,moveToReserveredfalse
2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Invocation exception, caused by:
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
com.cloud.agent.api.Answer cannot be cast to class
com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin:
4912
2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Unable
to complete AsyncJobVO {id:4915, userId: 4, accountId: 4, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9p
ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAlj
bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQA
EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-
AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01h
cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAE
SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAAE
AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAO
amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUd
C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHD
FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1Q
YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0,
status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed: null}, job
origin:4912
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
at
com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveIso(Con
figDriveNetworkElement.java:597)
at
com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUserdata(C
onfigDriveNetworkElement.java:225)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElemen
t(NetworkOrchestrator.java:1610)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(Ne
tworkOrchestrator.java:1936)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(Netwo
rkOrchestrator.java:1870)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:1201)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:5802)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
ccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
gMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
a:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
Impl.java:5967)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
ntextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
ultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
text(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
ext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
extRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
ManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaamF2YS5s
YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0f
Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2
YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAK
c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRF
eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdWQuYWdl
bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbnQuYXBp
LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3dlciBh
bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcmUgaW4g
dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNl
RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRh
CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH4ABUwA
DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFTAAK
bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2NvbS5j
bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbmZpZ0Ry
aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AAwBAAAA
4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAAZKcQB-
AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d29ya09y
Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW50cHBz
cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQB-AA5x
AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS5WaXJ0
dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQ
b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3EAfgAM
Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsdAAd
TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldAAHMTEu
MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAgAAACtw
dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbHQAIURl
bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADAIAAAI2
cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJXEAfgAm
c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpW
bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQAAF09x
AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS5WbVdv
cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icHBzcQB-
AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwu
QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5D
b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQu
Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxl
LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFu
YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdl
ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLmNsb3Vk
c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AEJ0AA9j
YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aENvbnRl
eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250
ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfgAOcQB-
ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRv
cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQB-AAwC
AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZh
cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFk
UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH4AJXEA
fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9y
JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLlRocmVh
ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW9ucyRF
bXB0eUxpc3R6uBe0PKee3gIAAHhweA
2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
Publish async job-4915 complete on message bus
2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Wake
up jobs related to job-4915
2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Update
db status for job-4915
2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Wake
up jobs joined with job-4915 and disjoin all subjobs created from job- 4915
2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Done
executing com.cloud.vm.VmWorkStart for job-4915
2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Remove
job-4915 from job monitoring
2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for command
[/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623 -U piotr -P
Seneka22 chassis power status]: [Chassis Power is on
].
2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623 -U
piotr -P Seneka22 chassis power status]: [Unable to Get Channel Cipher
Suites
Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
Error response 0xc1 from Get PICMG Properities
Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
Invalid completion code received: Invalid command
Discovered IPMB address 0x0
].
2021-11-17 14:47:27,805 DEBUG
[o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] (pool-2-thread-10:null)
(logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
successful and got the result [Chassis Power is on].
2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for command
[/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623 -U piotr -P
Seneka22 chassis power status]: [Chassis Power is on
].
2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623 -U
piotr -P Seneka22 chassis power status]: [Unable to Get Channel Cipher
Suites
Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
Error response 0xc1 from Get PICMG Properities
Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
Invalid completion code received: Invalid command
Discovered IPMB address 0x0
].
2021-11-17 14:47:27,806 DEBUG
[o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] (pool-2-thread-11:null)
(logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
successful and got the result [Chassis Power is on].
2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
exception while executing
org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
at
com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveIso(Con
figDriveNetworkElement.java:597)
at
com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUserdata(C
onfigDriveNetworkElement.java:225)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElemen
t(NetworkOrchestrator.java:1610)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(Ne
tworkOrchestrator.java:1936)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(Netwo
rkOrchestrator.java:1870)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:1201)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:5802)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
ccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
gMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
a:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
Impl.java:5967)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
ntextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
ultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
text(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
ext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
extRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
ManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete async
job-4912, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"er
rorcode":"530","errortext":"class com.cloud.agent.api.Answer cannot be cast
to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')"}
2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish async
job-4912 complete on message bus
2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up jobs
related to job-4912
2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db status
for job-4912
2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up jobs
joined with job-4912 and disjoin all subjobs created
Re: configdrive not working in 4.16
Posted by Wei ZHOU <us...@gmail.com>.
Hi Piotr,
sorry I asked you to check the log on the wrong host, since the command was
sent to SSVM (s-163-VM). One question, have you destroyed/recreated SSVM
after the upgrade ?
-Wei
On Thu, 18 Nov 2021 at 10:16, Piotr Pisz <pi...@piszki.pl> wrote:
> Hi Wei,
>
> Only this:
>
> 2021-11-17 14:47:22,940 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:17568bac) Seq 1-8430738502437568543:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is
> beating...","wait":"0","bypassHostMaintenance":"false"}}] }
> 2021-11-17 14:47:26,945 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:a9361d2a) Request:Seq
> 1-8430738502437568544: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 14:47:26,945 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:a9361d2a) Processing command:
> com.cloud.agent.api.CheckOnHostCommand
> 2021-11-17 14:47:26,945 DEBUG [kvm.resource.KVMHAChecker]
> (pool-147-thread-1:null) (logid:a9361d2a) Checking heart beat with
> KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
> 2021-11-17 14:47:26,946 DEBUG [kvm.resource.KVMHAChecker]
> (pool-147-thread-1:null) (logid:a9361d2a) Executing:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60
> 2021-11-17 14:47:26,946 DEBUG [kvm.resource.KVMHAChecker]
> (pool-147-thread-1:null) (logid:a9361d2a) Executing while with timeout :
> 360000
> 2021-11-17 14:47:26,955 DEBUG [kvm.resource.KVMHAChecker]
> (pool-147-thread-1:null) (logid:a9361d2a) Execution is successful.
> 2021-11-17 14:47:26,955 DEBUG [kvm.resource.KVMHAChecker]
> (pool-147-thread-1:null) (logid:a9361d2a) Checking heart beat with
> KVMHAChecker
> [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool:
> "nfs.piszki.lab"}].
> 2021-11-17 14:47:26,956 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:a9361d2a) Seq 1-8430738502437568544:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is
> beating...","wait":"0","bypassHostMaintenance":"false"}}] }
> 2021-11-17 14:47:27,508 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Request:Seq
> 1-8430738502437568545: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 14:47:27,508 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Processing command:
> com.cloud.agent.api.StopCommand
> 2021-11-17 14:47:27,508 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 14:47:27,509 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Can not find KVM connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 14:47:27,509 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Looking for libvirtd
> connection at: lxc:///
> 2021-11-17 14:47:27,509 INFO [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) No existing libvirtd
> connection found. Opening a new one
> 2021-11-17 14:47:27,510 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Can not find LXC connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 14:47:27,510 WARN [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Can not find a connection
> for Instance i-6-83-VM. Assuming the default connection.
> 2021-11-17 14:47:27,510 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> destroy_network_rules_for_vm --vmname i-6-83-VM
> 2021-11-17 14:47:27,512 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Executing while with timeout
> : 1800000
> 2021-11-17 14:47:27,646 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) Execution is successful.
> 2021-11-17 14:47:27,646 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) (logid:b59e4ee2) ebtables: No
> chain/target/match by that name
> ebtables: No chain/target/match by that name
>
> Regards,
> Piotr
>
>
> -----Original Message-----
> From: Wei ZHOU <us...@gmail.com>
> Sent: Thursday, November 18, 2021 7:10 AM
> To: piotr@piszki.pl
> Cc: users <us...@cloudstack.apache.org>
> Subject: Re: configdrive not working in 4.16
>
> Hi Piotr,
>
> Any error in agent.log around 2021-11-17 14:47:27 ?
>
> -Wei
>
> On Wed, 17 Nov 2021 at 20:56, Piotr Pisz <pi...@piszki.pl> wrote:
>
> > Hi Wei,
> >
> >
> >
> > This if from management.log:
> >
> >
> >
> > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}]
> > }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Details from executing class
> > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> config
> > drive at secondary storage
> >
> > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}]
> > }
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> > cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> > loader 'app')
> >
> > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Invocation exception, caused by:
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> > cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> > loader 'app')
> >
> > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> > com.cloud.agent.api.Answer cannot be cast to class
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer
> > and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of
> > loader 'app')
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> > cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> > loader 'app')
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> > cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> > loader 'app')
> >
> > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> async
> > job-4912, jobStatus: FAILED, resultCode: 530, result:
> >
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"class
> > com.cloud.agent.api.Answer cannot be cast to class
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer
> > and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of
> > loader 'app')"}
> >
> >
> >
> > Regards,
> >
> > Piotr
> >
> >
> >
> >
> >
> > *From:* Wei ZHOU <us...@gmail.com>
> > *Sent:* Wednesday, November 17, 2021 7:58 PM
> > *To:* users <us...@cloudstack.apache.org>; piotr@piszki.pl
> > *Subject:* Re: configdrive not working in 4.16
> >
> >
> >
> > Hi Piotr,
> >
> >
> >
> > Can you search HandleConfigDriveIsoCommand in agent.log and see if there
> > are any errors in when processing the command ?
> >
> >
> >
> > -Wei
> >
> >
> >
> > On Wed, 17 Nov 2021 at 16:25, Piotr Pisz <pi...@piszki.pl> wrote:
> >
> > Hi Wei,
> >
> > My KVM is CentOS 8.4 (upgrade from 4.15.2 without problems). VMs run on
> > Ceph/RBD, NFS is from Ganesha.
> >
> > 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> > (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> > KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
> > 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> > (pool-1486-thread-1:null) (logid:901784ad) Executing:
> > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> > nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> > 192.168.10.41 -r -t 60
> > 2021-11-17 16:20:20,075 DEBUG [kvm.resource.KVMHAChecker]
> > (pool-1486-thread-1:null) (logid:901784ad) Executing while with timeout :
> > 360000
> > 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> > (pool-1486-thread-1:null) (logid:901784ad) Execution is successful.
> > 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> > (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> > KVMHAChecker
> >
> [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> > -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> > 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====",
> pool:
> > "nfs.piszki.lab"}].
> > 2021-11-17 16:20:20,082 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) (logid:901784ad) Seq 1-4080261262397669854:
> > { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is
> > beating...","wait":"0","bypassHostMaintenance":"false"}}] }
> > 2021-11-17 16:20:23,687 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Request:Seq
> > 1-4080261262397669855: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> > Flags: 100011,
> >
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
> > }
> > 2021-11-17 16:20:23,688 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Processing command:
> > com.cloud.agent.api.StopCommand
> > 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> > connection at: qemu:///system
> > 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find KVM
> connection
> > for Instance: i-6-83-VM, continuing.
> > 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> > connection at: lxc:///
> > 2021-11-17 16:20:23,688 INFO [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) No existing libvirtd
> > connection found. Opening a new one
> > 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find LXC
> connection
> > for Instance: i-6-83-VM, continuing.
> > 2021-11-17 16:20:23,689 WARN [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find a connection
> > for Instance i-6-83-VM. Assuming the default connection.
> > 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> > connection at: qemu:///system
> > 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> > org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> pasującej
> > do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> > org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> pasującej
> > do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> > org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> pasującej
> > do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Executing:
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > destroy_network_rules_for_vm --vmname i-6-83-VM
> > 2021-11-17 16:20:23,691 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Executing while with
> timeout
> > : 1800000
> > 2021-11-17 16:20:23,823 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Execution is successful.
> > 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) ebtables: No
> > chain/target/match by that name
> > ebtables: No chain/target/match by that name
> > ebtables: No chain/target/match by that name
> > ebtables: No chain/target/match by that name
> > ebtables: No chain/target/match by that name
> > ebtables: No chain/target/match by that name
> > iptables: No chain/target/match by that name.
> > iptables: No chain/target/match by that name.
> > iptables: No chain/target/match by that name.
> > iptables: No chain/target/match by that name.
> > iptables: No chain/target/match by that name.
> > iptables: No chain/target/match by that name.
> > ipset v7.1: The set with the given name does not exist
> >
> > 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm :Nie
> > odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Try to stop the vm at
> first
> > 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to stop VM
> :i-6-83-VM
> > :
> > org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> > pasującej do nazwy „i-6-83-VM”
> > at org.libvirt.ErrorHandler.processError(Unknown Source)
> > at org.libvirt.ErrorHandler.processError(Unknown Source)
> > at org.libvirt.Connect.domainLookupByName(Unknown Source)
> > at
> >
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVMInternal(LibvirtComputingResource.java:3590)
> > at
> >
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3542)
> > at
> >
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:94)
> > at
> >
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:49)
> > at
> >
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
> > at
> >
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1768)
> > at com.cloud.agent.Agent.processRequest(Agent.java:661)
> > at
> > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079)
> > at com.cloud.utils.nio.Task.call(Task.java:83)
> > at com.cloud.utils.nio.Task.call(Task.java:29)
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > at
> >
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> > at
> >
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> > at java.base/java.lang.Thread.run(Thread.java:829)
> > 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm
> status:Nie
> > odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm
> status:Nie
> > odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm
> status:Nie
> > odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> > 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Can't get vm's status,
> > assume it's dead already
> > 2021-11-17 16:20:23,825 WARN [kvm.resource.LibvirtKvmAgentHook]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy script
> > '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not
> > available. Transformations will not be applied.
> > 2021-11-17 16:20:23,826 WARN [kvm.resource.LibvirtKvmAgentHook]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy scripting engine is
> > not initialized. Data transformation skipped.
> > 2021-11-17 16:20:23,826 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-3:null) (logid:706e0dbf) Seq 1-4080261262397669855:
> > { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> >
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
> > }
> > 2021-11-17 16:20:24,087 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-5:null) (logid:4f4315fe) Request:Seq
> > 1-4080261262397669856: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> > Flags: 100011,
> >
> [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}]
> > }
> >
> > Regards,
> > Piotr
> >
> >
> > -----Original Message-----
> > From: Wei ZHOU <us...@gmail.com>
> > Sent: Wednesday, November 17, 2021 4:15 PM
> > To: users <us...@cloudstack.apache.org>; ppisz@post.pl
> > Subject: Re: configdrive not working in 4.16
> >
> > Hi Piotr,
> >
> > Any errors in agent.log if you use KVM ?
> >
> > -Wei
> >
> > On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <pp...@post.pl> wrote:
> >
> > > Hi,
> > >
> > > I have upgraded to 4.16 and am having trouble booting my VM on L2
> > > network with ConfigDrive. I would have a request for help.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Piotr
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > > applying
> > > overprovisioning: 31200
> > >
> > > 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) We are allocating VM, increasing the used capacity of
> > > this
> > > host:1
> > >
> > > 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> > > 3900
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> > > 61916983296 ,Requested RAM: (4,00 GB) 4294967296
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) We are allocating VM to the last host again, so
> > > adjusting the reserved capacity if it is not less than required
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > > (4,00
> > > GB) 4294967296
> > >
> > > 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0,
> > > old
> > > reserved: 3900, actual total: 31200, total with overprovisioning:
> > > 31200; new used:3900, reserved:0; requested
> > > cpu:3900,alloc_from_last:true
> > >
> > > 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> > > bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> > > 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> > > requested mem: (4,00
> > > GB) 4294967296,alloc_from_last:true
> > >
> > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> > > support requested CPU: 1 and requested speed: 3900
> > >
> > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested
> > CPU:
> > > 3900 and requested RAM: (4,00 GB) 4294967296 ,
> cpuOverprovisioningFactor:
> > > 1.0
> > >
> > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > > applying
> > > overprovisioning: 31200
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) We need to allocate to the last host again, so
> > > checking if there is enough reserved capacity
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > > (4,00
> > > GB) 4294967296
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Host has enough CPU and RAM available
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> > > 3900,
> > > actual total: 31200, total with overprovisioning: 31200; requested
> > > cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> > > reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> > > requested
> > > mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> > > considerReservedCapacity?: true
> > >
> > > 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Network id=212 is already implemented
> > >
> > > 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > > id=212
> > >
> > > 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > > 1
> > >
> > > 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Asking ConfigDrive to prepare for
> > > Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
> > >
> > > 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
> > >
> > > 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=service-offering, content=KVM-RBD-Custom
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=availability-zone, content=Piszki Lab
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=local-hostname, content=Brooklyn
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=local-ipv4, content=192.168.40.243
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=public-ipv4, content=192.168.40.243
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=public-hostname, content=null
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=public-keys, content=null
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=cloud-identifier,
> > > content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=hypervisor-host-name, content=psc1.piszki.lab
> > >
> > > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Executing config drive creation command:
> > > /usr/bin/genisoimage -o
> > > /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> > > -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r
> > > -V
> > > config-2 /tmp/configdrive12981467783887013707
> > >
> > > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> > > /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> > > -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> > > config-2
> > > /tmp/configdrive12981467783887013707
> > >
> > > 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Executing while with timeout : 300000
> > >
> > > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Execution is successful.
> > >
> > > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> > > ISO-9660.
> > >
> > >
> > >
> > > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> > >
> > > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > > drive/
> > >
> > > i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.Nf
> > > sTO":{
> > >
> > > "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUns
> > > upport
> > >
> > >
> >
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> > > "false"}}] }
> > >
> > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> > > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > > Flags: 10,
> > > [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfull
> > > y
> > > saved config drive at secondary
> > > storage","wait":"0","bypassHostMaintenance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: ,
> MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> > >
> > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Details from executing class
> > > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> > > config drive at secondary storage
> > >
> > > 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> > > "83",
> > > name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > > type="User"} in Starting state
> > >
> > > 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> > > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
> > >
> > >
> >
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
> > >
> > > "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volu
> > > mesToD
> > >
> > > isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> > > "0","b
> > > ypassHostMaintenance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> > > Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> > > 10,
> > >
> > > [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypass
> > > HostMa
> > > intenance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> > > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> > > StopAnswer } }
> > >
> > > 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > > id=212
> > >
> > > 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > > -1
> > >
> > > 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Asking ConfigDrive to release
> > > NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-
> > > null]
> > >
> > > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
> > >
> > > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> > >
> > > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > > drive/
> > >
> > >
> >
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
> > >
> > > {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUn
> > > suppor
> > >
> > > tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaint
> > > enance
> > > ":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> > > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > > Flags: 10,
> > >
> > > [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHost
> > > Mainte
> > > nance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: ,
> MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> > >
> > > 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Unable to release some network resources for the VM
> > > VM instance {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> > >
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveI
> > > so(Con
> > > figDriveNetworkElement.java:661)
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriv
> > > eNetwo
> > > rkElement.java:167)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > > Nic(Ne
> > > tworkOrchestrator.java:2238)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > > (Netwo
> > > rkOrchestrator.java:2164)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachi
> > > neMana
> > > gerImpl.java:1836)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerIm
> > > pl.jav
> > > a:1826)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > > Manage
> > > rImpl.java:1373)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > > Manage
> > > rImpl.java:5802)
> > >
> > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > > ethodA
> > > ccessorImpl.java:62)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > > egatin
> > > gMethodAccessorImpl.java:43)
> > >
> > > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >
> > > at
> > >
> > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > > xy.jav
> > > a:107)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > > anager
> > > Impl.java:5967)
> > >
> > > at
> > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > > ontext
> > > (AsyncJobManagerImpl.java:620)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > > agedCo
> > > ntextRunnable.java:48)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > > l(Defa
> > > ultManagedContext.java:55)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > > ithCon
> > > text(DefaultManagedContext.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > > thCont
> > > ext(DefaultManagedContext.java:52)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > > edCont
> > > extRunnable.java:45)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > > yncJob
> > > ManagerImpl.java:568)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > > s.java
> > > :515)
> > >
> > > at
> > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > > Execut
> > > or.java:1128)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > > lExecu
> > > tor.java:628)
> > >
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Successfully released storage resources for the VM VM
> > > instance {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> > >
> > > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> > > instance
> > > {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > > type="User"} in Starting state
> > >
> > > 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> > > from [Starting] to [Stopped] with event [OperationFailed]. VM's
> original
> > host:
> > > Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> > > "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> > > null, host before state transition: Host {"id": "1", "name":
> > > "psc1.piszki.lab",
> > > "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
> > >
> > > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > > applying
> > > overprovisioning: 31200
> > >
> > > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and
> > > RAM after applying overprovisioning: (61,66 GB) 66211950592
> > >
> > > 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> > > actual total: 31200, total with overprovisioning: 31200; new used:
> > > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> > >
> > > 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> > > 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new
> > used:
> > > (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> > > false,moveToReserveredfalse
> > >
> > > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Invocation exception, caused by:
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> > > com.cloud.agent.api.Answer cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job
> > origin:
> > > 4912
> > >
> > > 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Unable to complete AsyncJobVO {id:4915, userId: 4, accountId: 4,
> > > instanceType:
> > > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> > >
> > > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAA
> > > Zhdm9p
> > >
> > > ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdD
> > > tMAAlj
> > >
> > > bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTm
> > > FtZXQA
> > >
> > > EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubm
> > > VycQB-
> > >
> > > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dG
> > > lsL01h
> > >
> > > cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJW
> > > drAgAE
> > >
> > > SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAA
> > > AAAAAE
> > >
> > > AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAX
> > > BzcgAO
> > >
> > > amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZX
> > > KGrJUd
> > >
> > > C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcA
> > > UH2sHD
> > >
> > > FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdA
> > > AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> > > cmdVersion: 0,
> > > status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> > > initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> > > lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> > > null}, job
> > > origin:4912
> > >
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > > so(Con
> > > figDriveNetworkElement.java:597)
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > > data(C
> > > onfigDriveNetworkElement.java:225)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > > Elemen
> > > t(NetworkOrchestrator.java:1610)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > > Nic(Ne
> > > tworkOrchestrator.java:1936)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > > (Netwo
> > > rkOrchestrator.java:1870)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > > Manage
> > > rImpl.java:1201)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > > Manage
> > > rImpl.java:5802)
> > >
> > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > > ethodA
> > > ccessorImpl.java:62)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > > egatin
> > > gMethodAccessorImpl.java:43)
> > >
> > > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >
> > > at
> > >
> > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > > xy.jav
> > > a:107)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > > anager
> > > Impl.java:5967)
> > >
> > > at
> > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > > ontext
> > > (AsyncJobManagerImpl.java:620)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > > agedCo
> > > ntextRunnable.java:48)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > > l(Defa
> > > ultManagedContext.java:55)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > > ithCon
> > > text(DefaultManagedContext.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > > thCont
> > > ext(DefaultManagedContext.java:52)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > > edCont
> > > extRunnable.java:45)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > > yncJob
> > > ManagerImpl.java:568)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > > s.java
> > > :515)
> > >
> > > at
> > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > > Execut
> > > or.java:1128)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > > lExecu
> > > tor.java:628)
> > >
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
> > >
> > > rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaam
> > > F2YS5s
> > >
> > > YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW
> > > 9u0P0f
> > >
> > > Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0AB
> > > VMamF2
> > >
> > > YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbm
> > > c7WwAK
> > >
> > > c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZX
> > > NzZWRF
> > >
> > > eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdW
> > > QuYWdl
> > >
> > > bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbn
> > > QuYXBp
> > >
> > > LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3
> > > dlciBh
> > >
> > > bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcm
> > > UgaW4g
> > >
> > > dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1
> > > RyYWNl
> > >
> > > RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW
> > > 1lbnRh
> > >
> > > CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH
> > > 4ABUwA
> > >
> > > DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfg
> > > AFTAAK
> > >
> > > bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2
> > > NvbS5j
> > >
> > > bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbm
> > > ZpZ0Ry
> > >
> > > aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AA
> > > wBAAAA
> > >
> > > 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAA
> > > ZKcQB-
> > >
> > > AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d2
> > > 9ya09y
> > >
> > > Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW
> > > 50cHBz
> > >
> > > cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQ
> > > B-AA5x
> > >
> > > AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS
> > > 5WaXJ0
> > >
> > > dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYX
> > > ZhdAAQ
> > >
> > > b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3
> > > EAfgAM
> > >
> > > Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbX
> > > BsdAAd
> > >
> > > TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldA
> > > AHMTEu
> > >
> > > MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAg
> > > AAACtw
> > >
> > > dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbH
> > > QAIURl
> > >
> > > bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADA
> > > IAAAI2
> > >
> > > cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJX
> > > EAfgAm
> > >
> > > c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveH
> > > l0ABpW
> > >
> > > bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQ
> > > AAF09x
> > >
> > > AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS
> > > 5WbVdv
> > >
> > > cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icH
> > > BzcQB-
> > >
> > > AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLm
> > > ltcGwu
> > >
> > > QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcn
> > > VuSW5D
> > >
> > > b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbm
> > > FnZWQu
> > >
> > > Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm
> > > 5hYmxl
> > >
> > > LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2
> > > subWFu
> > >
> > > YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TW
> > > FuYWdl
> > >
> > > ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLm
> > > Nsb3Vk
> > >
> > > c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AE
> > > J0AA9j
> > >
> > > YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aE
> > > NvbnRl
> > >
> > > eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC
> > > 5jb250
> > >
> > > ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfg
> > > AOcQB-
> > >
> > > ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeG
> > > VjdXRv
> > >
> > > cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQ
> > > B-AAwC
> > >
> > > AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay
> > > 5qYXZh
> > >
> > > cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVG
> > > hyZWFk
> > >
> > > UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH
> > > 4AJXEA
> > >
> > > fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZW
> > > N1dG9y
> > >
> > > JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLl
> > > RocmVh
> > >
> > > ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW
> > > 9ucyRF
> > > bXB0eUxpc3R6uBe0PKee3gIAAHhweA
> > >
> > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Publish async job-4915 complete on message bus
> > >
> > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Wake up jobs related to job-4915
> > >
> > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Update db status for job-4915
> > >
> > > 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Wake up jobs joined with job-4915 and disjoin all subjobs created from
> > > job- 4915
> > >
> > > 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Done executing com.cloud.vm.VmWorkStart for job-4915
> > >
> > > 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > > Remove
> > > job-4915 from job monitoring
> > >
> > > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > > -U piotr -P
> > > Seneka22 chassis power status]: [Chassis Power is on
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > > Cipher Suites
> > >
> > > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> > >
> > > Error response 0xc1 from Get PICMG Properities
> > >
> > > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> > >
> > > Invalid completion code received: Invalid command
> > >
> > > Discovered IPMB address 0x0
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,805 DEBUG
> > > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > > (pool-2-thread-10:null)
> > > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> > > successful and got the result [Chassis Power is on].
> > >
> > > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > > -U piotr -P
> > > Seneka22 chassis power status]: [Chassis Power is on
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > > Cipher Suites
> > >
> > > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> > >
> > > Error response 0xc1 from Get PICMG Properities
> > >
> > > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> > >
> > > Invalid completion code received: Invalid command
> > >
> > > Discovered IPMB address 0x0
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,806 DEBUG
> > > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > > (pool-2-thread-11:null)
> > > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> > > successful and got the result [Chassis Power is on].
> > >
> > > 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> > > exception while executing
> > > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
> > >
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > > so(Con
> > > figDriveNetworkElement.java:597)
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > > data(C
> > > onfigDriveNetworkElement.java:225)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > > Elemen
> > > t(NetworkOrchestrator.java:1610)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > > Nic(Ne
> > > tworkOrchestrator.java:1936)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > > (Netwo
> > > rkOrchestrator.java:1870)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > > Manage
> > > rImpl.java:1201)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > > Manage
> > > rImpl.java:5802)
> > >
> > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > > ethodA
> > > ccessorImpl.java:62)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > > egatin
> > > gMethodAccessorImpl.java:43)
> > >
> > > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >
> > > at
> > >
> > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > > xy.jav
> > > a:107)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > > anager
> > > Impl.java:5967)
> > >
> > > at
> > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > > ontext
> > > (AsyncJobManagerImpl.java:620)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > > agedCo
> > > ntextRunnable.java:48)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > > l(Defa
> > > ultManagedContext.java:55)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > > ithCon
> > > text(DefaultManagedContext.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > > thCont
> > > ext(DefaultManagedContext.java:52)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > > edCont
> > > extRunnable.java:45)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > > yncJob
> > > ManagerImpl.java:568)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > > s.java
> > > :515)
> > >
> > > at
> > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > > Execut
> > > or.java:1128)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > > lExecu
> > > tor.java:628)
> > >
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> > > async job-4912, jobStatus: FAILED, resultCode: 530, result:
> > >
> > > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> > > [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> > > cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')"}
> > >
> > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> > > async
> > > job-4912 complete on message bus
> > >
> > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > > jobs related to job-4912
> > >
> > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> > > status for job-4912
> > >
> > > 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > > jobs joined with job-4912 and disjoin all subjobs created
> > >
> > >
> >
> >
>
>
RE: configdrive not working in 4.16
Posted by Piotr Pisz <pi...@piszki.pl>.
Hi Wei,
Only this:
2021-11-17 14:47:22,940 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:17568bac) Seq 1-8430738502437568543: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is beating...","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:26,945 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:a9361d2a) Request:Seq 1-8430738502437568544: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:26,945 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:a9361d2a) Processing command: com.cloud.agent.api.CheckOnHostCommand
2021-11-17 14:47:26,945 DEBUG [kvm.resource.KVMHAChecker] (pool-147-thread-1:null) (logid:a9361d2a) Checking heart beat with KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
2021-11-17 14:47:26,946 DEBUG [kvm.resource.KVMHAChecker] (pool-147-thread-1:null) (logid:a9361d2a) Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h 192.168.10.41 -r -t 60
2021-11-17 14:47:26,946 DEBUG [kvm.resource.KVMHAChecker] (pool-147-thread-1:null) (logid:a9361d2a) Executing while with timeout : 360000
2021-11-17 14:47:26,955 DEBUG [kvm.resource.KVMHAChecker] (pool-147-thread-1:null) (logid:a9361d2a) Execution is successful.
2021-11-17 14:47:26,955 DEBUG [kvm.resource.KVMHAChecker] (pool-147-thread-1:null) (logid:a9361d2a) Checking heart beat with KVMHAChecker [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool: "nfs.piszki.lab"}].
2021-11-17 14:47:26,956 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:a9361d2a) Seq 1-8430738502437568544: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is beating...","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,508 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:b59e4ee2) Request:Seq 1-8430738502437568545: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,508 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:b59e4ee2) Processing command: com.cloud.agent.api.StopCommand
2021-11-17 14:47:27,508 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) Looking for libvirtd connection at: qemu:///system
2021-11-17 14:47:27,509 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) Can not find KVM connection for Instance: i-6-83-VM, continuing.
2021-11-17 14:47:27,509 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) Looking for libvirtd connection at: lxc:///
2021-11-17 14:47:27,509 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) No existing libvirtd connection found. Opening a new one
2021-11-17 14:47:27,510 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) Can not find LXC connection for Instance: i-6-83-VM, continuing.
2021-11-17 14:47:27,510 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) Can not find a connection for Instance i-6-83-VM. Assuming the default connection.
2021-11-17 14:47:27,510 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:b59e4ee2) Looking for libvirtd connection at: qemu:///system
2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 14:47:27,511 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname i-6-83-VM
2021-11-17 14:47:27,512 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) Executing while with timeout : 1800000
2021-11-17 14:47:27,646 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) Execution is successful.
2021-11-17 14:47:27,646 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:b59e4ee2) ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
Regards,
Piotr
-----Original Message-----
From: Wei ZHOU <us...@gmail.com>
Sent: Thursday, November 18, 2021 7:10 AM
To: piotr@piszki.pl
Cc: users <us...@cloudstack.apache.org>
Subject: Re: configdrive not working in 4.16
Hi Piotr,
Any error in agent.log around 2021-11-17 14:47:27 ?
-Wei
On Wed, 17 Nov 2021 at 20:56, Piotr Pisz <pi...@piszki.pl> wrote:
> Hi Wei,
>
>
>
> This if from management.log:
>
>
>
> 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Details from executing class
> com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved config
> drive at secondary storage
>
> 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Invocation exception, caused by:
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
> and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete async
> job-4912, jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
> and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')"}
>
>
>
> Regards,
>
> Piotr
>
>
>
>
>
> *From:* Wei ZHOU <us...@gmail.com>
> *Sent:* Wednesday, November 17, 2021 7:58 PM
> *To:* users <us...@cloudstack.apache.org>; piotr@piszki.pl
> *Subject:* Re: configdrive not working in 4.16
>
>
>
> Hi Piotr,
>
>
>
> Can you search HandleConfigDriveIsoCommand in agent.log and see if there
> are any errors in when processing the command ?
>
>
>
> -Wei
>
>
>
> On Wed, 17 Nov 2021 at 16:25, Piotr Pisz <pi...@piszki.pl> wrote:
>
> Hi Wei,
>
> My KVM is CentOS 8.4 (upgrade from 4.15.2 without problems). VMs run on
> Ceph/RBD, NFS is from Ganesha.
>
> 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
> 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Executing:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60
> 2021-11-17 16:20:20,075 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Executing while with timeout :
> 360000
> 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Execution is successful.
> 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> KVMHAChecker
> [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool:
> "nfs.piszki.lab"}].
> 2021-11-17 16:20:20,082 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:901784ad) Seq 1-4080261262397669854:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is
> beating...","wait":"0","bypassHostMaintenance":"false"}}] }
> 2021-11-17 16:20:23,687 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Request:Seq
> 1-4080261262397669855: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 16:20:23,688 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Processing command:
> com.cloud.agent.api.StopCommand
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find KVM connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: lxc:///
> 2021-11-17 16:20:23,688 INFO [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) No existing libvirtd
> connection found. Opening a new one
> 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find LXC connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 16:20:23,689 WARN [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find a connection
> for Instance i-6-83-VM. Assuming the default connection.
> 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> destroy_network_rules_for_vm --vmname i-6-83-VM
> 2021-11-17 16:20:23,691 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Executing while with timeout
> : 1800000
> 2021-11-17 16:20:23,823 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Execution is successful.
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) ebtables: No
> chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> ipset v7.1: The set with the given name does not exist
>
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm :Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Try to stop the vm at first
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to stop VM :i-6-83-VM
> :
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> pasującej do nazwy „i-6-83-VM”
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.Connect.domainLookupByName(Unknown Source)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVMInternal(LibvirtComputingResource.java:3590)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3542)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:94)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:49)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1768)
> at com.cloud.agent.Agent.processRequest(Agent.java:661)
> at
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079)
> at com.cloud.utils.nio.Task.call(Task.java:83)
> at com.cloud.utils.nio.Task.call(Task.java:29)
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can't get vm's status,
> assume it's dead already
> 2021-11-17 16:20:23,825 WARN [kvm.resource.LibvirtKvmAgentHook]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy script
> '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not
> available. Transformations will not be applied.
> 2021-11-17 16:20:23,826 WARN [kvm.resource.LibvirtKvmAgentHook]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy scripting engine is
> not initialized. Data transformation skipped.
> 2021-11-17 16:20:23,826 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Seq 1-4080261262397669855:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 16:20:24,087 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:4f4315fe) Request:Seq
> 1-4080261262397669856: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}]
> }
>
> Regards,
> Piotr
>
>
> -----Original Message-----
> From: Wei ZHOU <us...@gmail.com>
> Sent: Wednesday, November 17, 2021 4:15 PM
> To: users <us...@cloudstack.apache.org>; ppisz@post.pl
> Subject: Re: configdrive not working in 4.16
>
> Hi Piotr,
>
> Any errors in agent.log if you use KVM ?
>
> -Wei
>
> On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <pp...@post.pl> wrote:
>
> > Hi,
> >
> > I have upgraded to 4.16 and am having trouble booting my VM on L2
> > network with ConfigDrive. I would have a request for help.
> >
> >
> >
> > Regards,
> >
> > Piotr
> >
> >
> >
> >
> >
> >
> >
> > 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We are allocating VM, increasing the used capacity of
> > this
> > host:1
> >
> > 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> > 3900
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> > 61916983296 ,Requested RAM: (4,00 GB) 4294967296
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We are allocating VM to the last host again, so
> > adjusting the reserved capacity if it is not less than required
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > (4,00
> > GB) 4294967296
> >
> > 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0,
> > old
> > reserved: 3900, actual total: 31200, total with overprovisioning:
> > 31200; new used:3900, reserved:0; requested
> > cpu:3900,alloc_from_last:true
> >
> > 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> > bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> > 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> > requested mem: (4,00
> > GB) 4294967296,alloc_from_last:true
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> > support requested CPU: 1 and requested speed: 3900
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested
> CPU:
> > 3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
> > 1.0
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We need to allocate to the last host again, so
> > checking if there is enough reserved capacity
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > (4,00
> > GB) 4294967296
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Host has enough CPU and RAM available
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> > 3900,
> > actual total: 31200, total with overprovisioning: 31200; requested
> > cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> > reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> > requested
> > mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> > considerReservedCapacity?: true
> >
> > 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Network id=212 is already implemented
> >
> > 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > id=212
> >
> > 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > 1
> >
> > 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Asking ConfigDrive to prepare for
> > Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
> >
> > 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
> >
> > 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=service-offering, content=KVM-RBD-Custom
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=availability-zone, content=Piszki Lab
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=local-hostname, content=Brooklyn
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=local-ipv4, content=192.168.40.243
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-ipv4, content=192.168.40.243
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-hostname, content=null
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-keys, content=null
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=cloud-identifier,
> > content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=hypervisor-host-name, content=psc1.piszki.lab
> >
> > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing config drive creation command:
> > /usr/bin/genisoimage -o
> > /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> > -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r
> > -V
> > config-2 /tmp/configdrive12981467783887013707
> >
> > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> > /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> > -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> > config-2
> > /tmp/configdrive12981467783887013707
> >
> > 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing while with timeout : 300000
> >
> > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Execution is successful.
> >
> > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> > ISO-9660.
> >
> >
> >
> > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > drive/
> >
> > i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.Nf
> > sTO":{
> >
> > "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUns
> > upport
> >
> >
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> > "false"}}] }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfull
> > y
> > saved config drive at secondary
> > storage","wait":"0","bypassHostMaintenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Details from executing class
> > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> > config drive at secondary storage
> >
> > 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> > "83",
> > name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
> >
> >
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
> >
> > "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volu
> > mesToD
> >
> > isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> > "0","b
> > ypassHostMaintenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> > 10,
> >
> > [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypass
> > HostMa
> > intenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> > StopAnswer } }
> >
> > 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > id=212
> >
> > 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > -1
> >
> > 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Asking ConfigDrive to release
> > NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-
> > null]
> >
> > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
> >
> > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > drive/
> >
> >
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
> >
> > {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUn
> > suppor
> >
> > tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaint
> > enance
> > ":"false"}}] }
> >
> > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > Flags: 10,
> >
> > [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHost
> > Mainte
> > nance":"false"}}] }
> >
> > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> >
> > 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Unable to release some network resources for the VM
> > VM instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:661)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriv
> > eNetwo
> > rkElement.java:167)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > Nic(Ne
> > tworkOrchestrator.java:2238)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > (Netwo
> > rkOrchestrator.java:2164)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachi
> > neMana
> > gerImpl.java:1836)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerIm
> > pl.jav
> > a:1826)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1373)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Successfully released storage resources for the VM VM
> > instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> > instance
> > {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> > from [Starting] to [Stopped] with event [OperationFailed]. VM's original
> host:
> > Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> > "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> > null, host before state transition: Host {"id": "1", "name":
> > "psc1.piszki.lab",
> > "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
> >
> > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and
> > RAM after applying overprovisioning: (61,66 GB) 66211950592
> >
> > 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> > actual total: 31200, total with overprovisioning: 31200; new used:
> > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> >
> > 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> > 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new
> used:
> > (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> > false,moveToReserveredfalse
> >
> > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Invocation exception, caused by:
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> > com.cloud.agent.api.Answer cannot be cast to class
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job
> origin:
> > 4912
> >
> > 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Unable to complete AsyncJobVO {id:4915, userId: 4, accountId: 4,
> > instanceType:
> > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> >
> > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAA
> > Zhdm9p
> >
> > ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdD
> > tMAAlj
> >
> > bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTm
> > FtZXQA
> >
> > EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubm
> > VycQB-
> >
> > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dG
> > lsL01h
> >
> > cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJW
> > drAgAE
> >
> > SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAA
> > AAAAAE
> >
> > AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAX
> > BzcgAO
> >
> > amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZX
> > KGrJUd
> >
> > C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcA
> > UH2sHD
> >
> > FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdA
> > AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> > cmdVersion: 0,
> > status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> > initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> > lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> > null}, job
> > origin:4912
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:597)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > data(C
> > onfigDriveNetworkElement.java:225)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Elemen
> > t(NetworkOrchestrator.java:1610)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Nic(Ne
> > tworkOrchestrator.java:1936)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > (Netwo
> > rkOrchestrator.java:1870)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1201)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
> >
> > rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaam
> > F2YS5s
> >
> > YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW
> > 9u0P0f
> >
> > Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0AB
> > VMamF2
> >
> > YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbm
> > c7WwAK
> >
> > c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZX
> > NzZWRF
> >
> > eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdW
> > QuYWdl
> >
> > bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbn
> > QuYXBp
> >
> > LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3
> > dlciBh
> >
> > bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcm
> > UgaW4g
> >
> > dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1
> > RyYWNl
> >
> > RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW
> > 1lbnRh
> >
> > CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH
> > 4ABUwA
> >
> > DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfg
> > AFTAAK
> >
> > bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2
> > NvbS5j
> >
> > bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbm
> > ZpZ0Ry
> >
> > aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AA
> > wBAAAA
> >
> > 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAA
> > ZKcQB-
> >
> > AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d2
> > 9ya09y
> >
> > Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW
> > 50cHBz
> >
> > cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQ
> > B-AA5x
> >
> > AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS
> > 5WaXJ0
> >
> > dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYX
> > ZhdAAQ
> >
> > b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3
> > EAfgAM
> >
> > Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbX
> > BsdAAd
> >
> > TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldA
> > AHMTEu
> >
> > MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAg
> > AAACtw
> >
> > dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbH
> > QAIURl
> >
> > bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADA
> > IAAAI2
> >
> > cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJX
> > EAfgAm
> >
> > c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveH
> > l0ABpW
> >
> > bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQ
> > AAF09x
> >
> > AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS
> > 5WbVdv
> >
> > cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icH
> > BzcQB-
> >
> > AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLm
> > ltcGwu
> >
> > QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcn
> > VuSW5D
> >
> > b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbm
> > FnZWQu
> >
> > Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm
> > 5hYmxl
> >
> > LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2
> > subWFu
> >
> > YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TW
> > FuYWdl
> >
> > ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLm
> > Nsb3Vk
> >
> > c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AE
> > J0AA9j
> >
> > YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aE
> > NvbnRl
> >
> > eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC
> > 5jb250
> >
> > ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfg
> > AOcQB-
> >
> > ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeG
> > VjdXRv
> >
> > cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQ
> > B-AAwC
> >
> > AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay
> > 5qYXZh
> >
> > cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVG
> > hyZWFk
> >
> > UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH
> > 4AJXEA
> >
> > fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZW
> > N1dG9y
> >
> > JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLl
> > RocmVh
> >
> > ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW
> > 9ucyRF
> > bXB0eUxpc3R6uBe0PKee3gIAAHhweA
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Publish async job-4915 complete on message bus
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Wake up jobs related to job-4915
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Update db status for job-4915
> >
> > 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Wake up jobs joined with job-4915 and disjoin all subjobs created from
> > job- 4915
> >
> > 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Done executing com.cloud.vm.VmWorkStart for job-4915
> >
> > 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Remove
> > job-4915 from job monitoring
> >
> > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > -U piotr -P
> > Seneka22 chassis power status]: [Chassis Power is on
> >
> > ].
> >
> > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > Cipher Suites
> >
> > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> >
> > Error response 0xc1 from Get PICMG Properities
> >
> > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> >
> > Invalid completion code received: Invalid command
> >
> > Discovered IPMB address 0x0
> >
> > ].
> >
> > 2021-11-17 14:47:27,805 DEBUG
> > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > (pool-2-thread-10:null)
> > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> > successful and got the result [Chassis Power is on].
> >
> > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > -U piotr -P
> > Seneka22 chassis power status]: [Chassis Power is on
> >
> > ].
> >
> > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > Cipher Suites
> >
> > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> >
> > Error response 0xc1 from Get PICMG Properities
> >
> > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> >
> > Invalid completion code received: Invalid command
> >
> > Discovered IPMB address 0x0
> >
> > ].
> >
> > 2021-11-17 14:47:27,806 DEBUG
> > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > (pool-2-thread-11:null)
> > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> > successful and got the result [Chassis Power is on].
> >
> > 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> > exception while executing
> > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:597)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > data(C
> > onfigDriveNetworkElement.java:225)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Elemen
> > t(NetworkOrchestrator.java:1610)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Nic(Ne
> > tworkOrchestrator.java:1936)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > (Netwo
> > rkOrchestrator.java:1870)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1201)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> > async job-4912, jobStatus: FAILED, resultCode: 530, result:
> >
> > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> > [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> > cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')"}
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> > async
> > job-4912 complete on message bus
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > jobs related to job-4912
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> > status for job-4912
> >
> > 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > jobs joined with job-4912 and disjoin all subjobs created
> >
> >
>
>
Re: configdrive not working in 4.16
Posted by Wei ZHOU <us...@gmail.com>.
Hi Piotr,
Any error in agent.log around 2021-11-17 14:47:27 ?
-Wei
On Wed, 17 Nov 2021 at 20:56, Piotr Pisz <pi...@piszki.pl> wrote:
> Hi Wei,
>
>
>
> This if from management.log:
>
>
>
> 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Details from executing class
> com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved config
> drive at secondary storage
>
> 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Invocation exception, caused by:
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
> and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete async
> job-4912, jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
> and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')"}
>
>
>
> Regards,
>
> Piotr
>
>
>
>
>
> *From:* Wei ZHOU <us...@gmail.com>
> *Sent:* Wednesday, November 17, 2021 7:58 PM
> *To:* users <us...@cloudstack.apache.org>; piotr@piszki.pl
> *Subject:* Re: configdrive not working in 4.16
>
>
>
> Hi Piotr,
>
>
>
> Can you search HandleConfigDriveIsoCommand in agent.log and see if there
> are any errors in when processing the command ?
>
>
>
> -Wei
>
>
>
> On Wed, 17 Nov 2021 at 16:25, Piotr Pisz <pi...@piszki.pl> wrote:
>
> Hi Wei,
>
> My KVM is CentOS 8.4 (upgrade from 4.15.2 without problems). VMs run on
> Ceph/RBD, NFS is from Ganesha.
>
> 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
> 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Executing:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60
> 2021-11-17 16:20:20,075 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Executing while with timeout :
> 360000
> 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Execution is successful.
> 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> KVMHAChecker
> [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool:
> "nfs.piszki.lab"}].
> 2021-11-17 16:20:20,082 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:901784ad) Seq 1-4080261262397669854:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is
> beating...","wait":"0","bypassHostMaintenance":"false"}}] }
> 2021-11-17 16:20:23,687 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Request:Seq
> 1-4080261262397669855: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 16:20:23,688 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Processing command:
> com.cloud.agent.api.StopCommand
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find KVM connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: lxc:///
> 2021-11-17 16:20:23,688 INFO [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) No existing libvirtd
> connection found. Opening a new one
> 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find LXC connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 16:20:23,689 WARN [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find a connection
> for Instance i-6-83-VM. Assuming the default connection.
> 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> destroy_network_rules_for_vm --vmname i-6-83-VM
> 2021-11-17 16:20:23,691 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Executing while with timeout
> : 1800000
> 2021-11-17 16:20:23,823 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Execution is successful.
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) ebtables: No
> chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> ipset v7.1: The set with the given name does not exist
>
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm :Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Try to stop the vm at first
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to stop VM :i-6-83-VM
> :
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> pasującej do nazwy „i-6-83-VM”
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.Connect.domainLookupByName(Unknown Source)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVMInternal(LibvirtComputingResource.java:3590)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3542)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:94)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:49)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1768)
> at com.cloud.agent.Agent.processRequest(Agent.java:661)
> at
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079)
> at com.cloud.utils.nio.Task.call(Task.java:83)
> at com.cloud.utils.nio.Task.call(Task.java:29)
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can't get vm's status,
> assume it's dead already
> 2021-11-17 16:20:23,825 WARN [kvm.resource.LibvirtKvmAgentHook]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy script
> '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not
> available. Transformations will not be applied.
> 2021-11-17 16:20:23,826 WARN [kvm.resource.LibvirtKvmAgentHook]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy scripting engine is
> not initialized. Data transformation skipped.
> 2021-11-17 16:20:23,826 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Seq 1-4080261262397669855:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 16:20:24,087 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:4f4315fe) Request:Seq
> 1-4080261262397669856: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}]
> }
>
> Regards,
> Piotr
>
>
> -----Original Message-----
> From: Wei ZHOU <us...@gmail.com>
> Sent: Wednesday, November 17, 2021 4:15 PM
> To: users <us...@cloudstack.apache.org>; ppisz@post.pl
> Subject: Re: configdrive not working in 4.16
>
> Hi Piotr,
>
> Any errors in agent.log if you use KVM ?
>
> -Wei
>
> On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <pp...@post.pl> wrote:
>
> > Hi,
> >
> > I have upgraded to 4.16 and am having trouble booting my VM on L2
> > network with ConfigDrive. I would have a request for help.
> >
> >
> >
> > Regards,
> >
> > Piotr
> >
> >
> >
> >
> >
> >
> >
> > 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We are allocating VM, increasing the used capacity of
> > this
> > host:1
> >
> > 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> > 3900
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> > 61916983296 ,Requested RAM: (4,00 GB) 4294967296
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We are allocating VM to the last host again, so
> > adjusting the reserved capacity if it is not less than required
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > (4,00
> > GB) 4294967296
> >
> > 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0,
> > old
> > reserved: 3900, actual total: 31200, total with overprovisioning:
> > 31200; new used:3900, reserved:0; requested
> > cpu:3900,alloc_from_last:true
> >
> > 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> > bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> > 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> > requested mem: (4,00
> > GB) 4294967296,alloc_from_last:true
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> > support requested CPU: 1 and requested speed: 3900
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested
> CPU:
> > 3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
> > 1.0
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We need to allocate to the last host again, so
> > checking if there is enough reserved capacity
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > (4,00
> > GB) 4294967296
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Host has enough CPU and RAM available
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> > 3900,
> > actual total: 31200, total with overprovisioning: 31200; requested
> > cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> > reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> > requested
> > mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> > considerReservedCapacity?: true
> >
> > 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Network id=212 is already implemented
> >
> > 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > id=212
> >
> > 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > 1
> >
> > 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Asking ConfigDrive to prepare for
> > Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
> >
> > 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
> >
> > 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=service-offering, content=KVM-RBD-Custom
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=availability-zone, content=Piszki Lab
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=local-hostname, content=Brooklyn
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=local-ipv4, content=192.168.40.243
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-ipv4, content=192.168.40.243
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-hostname, content=null
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-keys, content=null
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=cloud-identifier,
> > content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=hypervisor-host-name, content=psc1.piszki.lab
> >
> > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing config drive creation command:
> > /usr/bin/genisoimage -o
> > /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> > -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r
> > -V
> > config-2 /tmp/configdrive12981467783887013707
> >
> > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> > /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> > -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> > config-2
> > /tmp/configdrive12981467783887013707
> >
> > 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing while with timeout : 300000
> >
> > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Execution is successful.
> >
> > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> > ISO-9660.
> >
> >
> >
> > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > drive/
> >
> > i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.Nf
> > sTO":{
> >
> > "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUns
> > upport
> >
> >
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> > "false"}}] }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfull
> > y
> > saved config drive at secondary
> > storage","wait":"0","bypassHostMaintenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Details from executing class
> > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> > config drive at secondary storage
> >
> > 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> > "83",
> > name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
> >
> >
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
> >
> > "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volu
> > mesToD
> >
> > isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> > "0","b
> > ypassHostMaintenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> > 10,
> >
> > [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypass
> > HostMa
> > intenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> > StopAnswer } }
> >
> > 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > id=212
> >
> > 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > -1
> >
> > 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Asking ConfigDrive to release
> > NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-
> > null]
> >
> > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
> >
> > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > drive/
> >
> >
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
> >
> > {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUn
> > suppor
> >
> > tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaint
> > enance
> > ":"false"}}] }
> >
> > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > Flags: 10,
> >
> > [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHost
> > Mainte
> > nance":"false"}}] }
> >
> > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> >
> > 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Unable to release some network resources for the VM
> > VM instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:661)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriv
> > eNetwo
> > rkElement.java:167)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > Nic(Ne
> > tworkOrchestrator.java:2238)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > (Netwo
> > rkOrchestrator.java:2164)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachi
> > neMana
> > gerImpl.java:1836)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerIm
> > pl.jav
> > a:1826)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1373)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Successfully released storage resources for the VM VM
> > instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> > instance
> > {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> > from [Starting] to [Stopped] with event [OperationFailed]. VM's original
> host:
> > Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> > "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> > null, host before state transition: Host {"id": "1", "name":
> > "psc1.piszki.lab",
> > "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
> >
> > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and
> > RAM after applying overprovisioning: (61,66 GB) 66211950592
> >
> > 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> > actual total: 31200, total with overprovisioning: 31200; new used:
> > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> >
> > 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> > 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new
> used:
> > (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> > false,moveToReserveredfalse
> >
> > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Invocation exception, caused by:
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> > com.cloud.agent.api.Answer cannot be cast to class
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job
> origin:
> > 4912
> >
> > 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Unable to complete AsyncJobVO {id:4915, userId: 4, accountId: 4,
> > instanceType:
> > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> >
> > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAA
> > Zhdm9p
> >
> > ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdD
> > tMAAlj
> >
> > bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTm
> > FtZXQA
> >
> > EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubm
> > VycQB-
> >
> > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dG
> > lsL01h
> >
> > cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJW
> > drAgAE
> >
> > SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAA
> > AAAAAE
> >
> > AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAX
> > BzcgAO
> >
> > amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZX
> > KGrJUd
> >
> > C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcA
> > UH2sHD
> >
> > FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdA
> > AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> > cmdVersion: 0,
> > status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> > initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> > lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> > null}, job
> > origin:4912
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:597)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > data(C
> > onfigDriveNetworkElement.java:225)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Elemen
> > t(NetworkOrchestrator.java:1610)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Nic(Ne
> > tworkOrchestrator.java:1936)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > (Netwo
> > rkOrchestrator.java:1870)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1201)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
> >
> > rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaam
> > F2YS5s
> >
> > YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW
> > 9u0P0f
> >
> > Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0AB
> > VMamF2
> >
> > YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbm
> > c7WwAK
> >
> > c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZX
> > NzZWRF
> >
> > eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdW
> > QuYWdl
> >
> > bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbn
> > QuYXBp
> >
> > LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3
> > dlciBh
> >
> > bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcm
> > UgaW4g
> >
> > dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1
> > RyYWNl
> >
> > RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW
> > 1lbnRh
> >
> > CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH
> > 4ABUwA
> >
> > DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfg
> > AFTAAK
> >
> > bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2
> > NvbS5j
> >
> > bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbm
> > ZpZ0Ry
> >
> > aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AA
> > wBAAAA
> >
> > 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAA
> > ZKcQB-
> >
> > AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d2
> > 9ya09y
> >
> > Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW
> > 50cHBz
> >
> > cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQ
> > B-AA5x
> >
> > AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS
> > 5WaXJ0
> >
> > dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYX
> > ZhdAAQ
> >
> > b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3
> > EAfgAM
> >
> > Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbX
> > BsdAAd
> >
> > TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldA
> > AHMTEu
> >
> > MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAg
> > AAACtw
> >
> > dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbH
> > QAIURl
> >
> > bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADA
> > IAAAI2
> >
> > cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJX
> > EAfgAm
> >
> > c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveH
> > l0ABpW
> >
> > bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQ
> > AAF09x
> >
> > AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS
> > 5WbVdv
> >
> > cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icH
> > BzcQB-
> >
> > AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLm
> > ltcGwu
> >
> > QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcn
> > VuSW5D
> >
> > b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbm
> > FnZWQu
> >
> > Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm
> > 5hYmxl
> >
> > LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2
> > subWFu
> >
> > YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TW
> > FuYWdl
> >
> > ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLm
> > Nsb3Vk
> >
> > c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AE
> > J0AA9j
> >
> > YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aE
> > NvbnRl
> >
> > eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC
> > 5jb250
> >
> > ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfg
> > AOcQB-
> >
> > ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeG
> > VjdXRv
> >
> > cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQ
> > B-AAwC
> >
> > AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay
> > 5qYXZh
> >
> > cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVG
> > hyZWFk
> >
> > UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH
> > 4AJXEA
> >
> > fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZW
> > N1dG9y
> >
> > JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLl
> > RocmVh
> >
> > ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW
> > 9ucyRF
> > bXB0eUxpc3R6uBe0PKee3gIAAHhweA
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Publish async job-4915 complete on message bus
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Wake up jobs related to job-4915
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Update db status for job-4915
> >
> > 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Wake up jobs joined with job-4915 and disjoin all subjobs created from
> > job- 4915
> >
> > 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Done executing com.cloud.vm.VmWorkStart for job-4915
> >
> > 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Remove
> > job-4915 from job monitoring
> >
> > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > -U piotr -P
> > Seneka22 chassis power status]: [Chassis Power is on
> >
> > ].
> >
> > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > Cipher Suites
> >
> > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> >
> > Error response 0xc1 from Get PICMG Properities
> >
> > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> >
> > Invalid completion code received: Invalid command
> >
> > Discovered IPMB address 0x0
> >
> > ].
> >
> > 2021-11-17 14:47:27,805 DEBUG
> > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > (pool-2-thread-10:null)
> > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> > successful and got the result [Chassis Power is on].
> >
> > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > -U piotr -P
> > Seneka22 chassis power status]: [Chassis Power is on
> >
> > ].
> >
> > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > Cipher Suites
> >
> > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> >
> > Error response 0xc1 from Get PICMG Properities
> >
> > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> >
> > Invalid completion code received: Invalid command
> >
> > Discovered IPMB address 0x0
> >
> > ].
> >
> > 2021-11-17 14:47:27,806 DEBUG
> > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > (pool-2-thread-11:null)
> > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> > successful and got the result [Chassis Power is on].
> >
> > 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> > exception while executing
> > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:597)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > data(C
> > onfigDriveNetworkElement.java:225)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Elemen
> > t(NetworkOrchestrator.java:1610)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Nic(Ne
> > tworkOrchestrator.java:1936)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > (Netwo
> > rkOrchestrator.java:1870)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1201)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> > async job-4912, jobStatus: FAILED, resultCode: 530, result:
> >
> > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> > [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> > cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')"}
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> > async
> > job-4912 complete on message bus
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > jobs related to job-4912
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> > status for job-4912
> >
> > 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > jobs joined with job-4912 and disjoin all subjobs created
> >
> >
>
>
RE: configdrive not working in 4.16
Posted by Piotr Pisz <pi...@piszki.pl>.
Hi Wei,
This if from management.log:
2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request] (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId: 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl] (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) (logid:b59e4ee2) Details from executing class com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved config drive at secondary storage
2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request] (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId: 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupportedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance":"false"}}] }
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of loader 'app')
2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) (logid:b59e4ee2) Invocation exception, caused by: java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of loader 'app')
2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of loader 'app')
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of loader 'app')
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of loader 'app')
2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete async job-4912, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of loader 'app')"}
Regards,
Piotr
From: Wei ZHOU <us...@gmail.com>
Sent: Wednesday, November 17, 2021 7:58 PM
To: users <us...@cloudstack.apache.org>; piotr@piszki.pl
Subject: Re: configdrive not working in 4.16
Hi Piotr,
Can you search HandleConfigDriveIsoCommand in agent.log and see if there are any errors in when processing the command ?
-Wei
On Wed, 17 Nov 2021 at 16:25, Piotr Pisz <piotr@piszki.pl <ma...@piszki.pl> > wrote:
Hi Wei,
My KVM is CentOS 8.4 (upgrade from 4.15.2 without problems). VMs run on Ceph/RBD, NFS is from Ganesha.
2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h 192.168.10.41 -r -t 60
2021-11-17 16:20:20,075 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Executing while with timeout : 360000
2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Execution is successful.
2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with KVMHAChecker [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool: "nfs.piszki.lab"}].
2021-11-17 16:20:20,082 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:901784ad) Seq 1-4080261262397669854: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is beating...","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 16:20:23,687 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:706e0dbf) Request:Seq 1-4080261262397669855: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 16:20:23,688 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:706e0dbf) Processing command: com.cloud.agent.api.StopCommand
2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd connection at: qemu:///system
2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find KVM connection for Instance: i-6-83-VM, continuing.
2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd connection at: lxc:///
2021-11-17 16:20:23,688 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) No existing libvirtd connection found. Opening a new one
2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find LXC connection for Instance: i-6-83-VM, continuing.
2021-11-17 16:20:23,689 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find a connection for Instance i-6-83-VM. Assuming the default connection.
2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd connection at: qemu:///system
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname i-6-83-VM
2021-11-17 16:20:23,691 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Executing while with timeout : 1800000
2021-11-17 16:20:23,823 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Execution is successful.
2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
ipset v7.1: The set with the given name does not exist
2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm :Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Try to stop the vm at first
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to stop VM :i-6-83-VM :
org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.Connect.domainLookupByName(Unknown Source)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVMInternal(LibvirtComputingResource.java:3590)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3542)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:94)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:49)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1768)
at com.cloud.agent.Agent.processRequest(Agent.java:661)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Can't get vm's status, assume it's dead already
2021-11-17 16:20:23,825 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
2021-11-17 16:20:23,826 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy scripting engine is not initialized. Data transformation skipped.
2021-11-17 16:20:23,826 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:706e0dbf) Seq 1-4080261262397669855: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 16:20:24,087 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:4f4315fe) Request:Seq 1-4080261262397669856: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}] }
Regards,
Piotr
-----Original Message-----
From: Wei ZHOU <ustcweizhou@gmail.com <ma...@gmail.com> >
Sent: Wednesday, November 17, 2021 4:15 PM
To: users <users@cloudstack.apache.org <ma...@cloudstack.apache.org> >; ppisz@post.pl <ma...@post.pl>
Subject: Re: configdrive not working in 4.16
Hi Piotr,
Any errors in agent.log if you use KVM ?
-Wei
On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <ppisz@post.pl <ma...@post.pl> > wrote:
> Hi,
>
> I have upgraded to 4.16 and am having trouble booting my VM on L2
> network with ConfigDrive. I would have a request for help.
>
>
>
> Regards,
>
> Piotr
>
>
>
>
>
>
>
> 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We are allocating VM, increasing the used capacity of
> this
> host:1
>
> 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> 3900
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> 61916983296 ,Requested RAM: (4,00 GB) 4294967296
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We are allocating VM to the last host again, so
> adjusting the reserved capacity if it is not less than required
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> (4,00
> GB) 4294967296
>
> 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0,
> old
> reserved: 3900, actual total: 31200, total with overprovisioning:
> 31200; new used:3900, reserved:0; requested
> cpu:3900,alloc_from_last:true
>
> 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> requested mem: (4,00
> GB) 4294967296,alloc_from_last:true
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> support requested CPU: 1 and requested speed: 3900
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested CPU:
> 3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
> 1.0
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We need to allocate to the last host again, so
> checking if there is enough reserved capacity
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> (4,00
> GB) 4294967296
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Host has enough CPU and RAM available
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> 3900,
> actual total: 31200, total with overprovisioning: 31200; requested
> cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> requested
> mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> considerReservedCapacity?: true
>
> 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Network id=212 is already implemented
>
> 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> id=212
>
> 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Changing active number of nics for network id=212 on
> 1
>
> 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Asking ConfigDrive to prepare for
> Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
>
> 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
>
> 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=service-offering, content=KVM-RBD-Custom
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=availability-zone, content=Piszki Lab
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=local-hostname, content=Brooklyn
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=local-ipv4, content=192.168.40.243
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-ipv4, content=192.168.40.243
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-hostname, content=null
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-keys, content=null
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=cloud-identifier,
> content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=hypervisor-host-name, content=psc1.piszki.lab
>
> 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing config drive creation command:
> /usr/bin/genisoimage -o
> /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r
> -V
> config-2 /tmp/configdrive12981467783887013707
>
> 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> config-2
> /tmp/configdrive12981467783887013707
>
> 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing while with timeout : 300000
>
> 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Execution is successful.
>
> 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> ISO-9660.
>
>
>
> 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> drive/
>
> i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.Nf <http://com.cloud.agent.api.to.Nf>
> sTO":{
>
> "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUns
> upport
>
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> "false"}}] }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfull
> y
> saved config drive at secondary
> storage","wait":"0","bypassHostMaintenance":"false"}}] }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Details from executing class
> com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> config drive at secondary storage
>
> 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> "83",
> name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> type="User"} in Starting state
>
> 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
>
> "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volu
> mesToD
>
> isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> "0","b
> ypassHostMaintenance":"false"}}] }
>
> 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> 10,
>
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypass
> HostMa
> intenance":"false"}}] }
>
> 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> StopAnswer } }
>
> 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> id=212
>
> 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Changing active number of nics for network id=212 on
> -1
>
> 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Asking ConfigDrive to release
> NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-
> null]
>
> 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
>
> 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> drive/
>
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
>
> {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUn
> suppor
>
> tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaint
> enance
> ":"false"}}] }
>
> 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> Flags: 10,
>
> [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHost
> Mainte
> nance":"false"}}] }
>
> 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
>
> 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Unable to release some network resources for the VM
> VM instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveI
> so(Con
> figDriveNetworkElement.java:661)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriv
> eNetwo
> rkElement.java:167)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> Nic(Ne
> tworkOrchestrator.java:2238)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> (Netwo
> rkOrchestrator.java:2164)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachi
> neMana
> gerImpl.java:1836)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerIm
> pl.jav
> a:1826)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:1373)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> ethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> egatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> xy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> anager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> ontext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> agedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> l(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> ithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> thCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> edCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> yncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> s.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Execut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Successfully released storage resources for the VM VM
> instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
>
> 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> instance
> {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> type="User"} in Starting state
>
> 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> from [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
> Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> null, host before state transition: Host {"id": "1", "name":
> "psc1.piszki.lab",
> "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
>
> 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and
> RAM after applying overprovisioning: (61,66 GB) 66211950592
>
> 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> actual total: 31200, total with overprovisioning: 31200; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>
> 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new used:
> (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> false,moveToReserveredfalse
>
> 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Invocation exception, caused by:
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin:
> 4912
>
> 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Unable to complete AsyncJobVO {id:4915, userId: 4, accountId: 4,
> instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAA
> Zhdm9p
>
> ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdD
> tMAAlj
>
> bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTm
> FtZXQA
>
> EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubm
> VycQB-
>
> AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dG
> lsL01h
>
> cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJW
> drAgAE
>
> SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAA
> AAAAAE
>
> AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAX
> BzcgAO
>
> amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZX
> KGrJUd
>
> C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcA
> UH2sHD
>
> FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdA
> AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0,
> status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> null}, job
> origin:4912
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> so(Con
> figDriveNetworkElement.java:597)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> data(C
> onfigDriveNetworkElement.java:225)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Elemen
> t(NetworkOrchestrator.java:1610)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Nic(Ne
> tworkOrchestrator.java:1936)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> (Netwo
> rkOrchestrator.java:1870)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:1201)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> ethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> egatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> xy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> anager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> ontext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> agedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> l(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> ithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> thCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> edCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> yncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> s.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Execut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaam
> F2YS5s
>
> YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW
> 9u0P0f
>
> Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0AB
> VMamF2
>
> YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbm
> c7WwAK
>
> c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZX
> NzZWRF
>
> eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdW
> QuYWdl
>
> bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbn
> QuYXBp
>
> LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3
> dlciBh
>
> bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcm
> UgaW4g
>
> dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1
> RyYWNl
>
> RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW
> 1lbnRh
>
> CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH
> 4ABUwA
>
> DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfg
> AFTAAK
>
> bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2
> NvbS5j
>
> bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbm
> ZpZ0Ry
>
> aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AA
> wBAAAA
>
> 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAA
> ZKcQB-
>
> AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d2
> 9ya09y
>
> Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW
> 50cHBz
>
> cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQ
> B-AA5x
>
> AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS
> 5WaXJ0
>
> dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYX
> ZhdAAQ
>
> b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3
> EAfgAM
>
> Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbX
> BsdAAd
>
> TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldA
> AHMTEu
>
> MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAg
> AAACtw
>
> dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbH
> QAIURl
>
> bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADA
> IAAAI2
>
> cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJX
> EAfgAm
>
> c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveH
> l0ABpW
>
> bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQ
> AAF09x
>
> AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS
> 5WbVdv
>
> cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icH
> BzcQB-
>
> AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLm
> ltcGwu
>
> QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcn
> VuSW5D
>
> b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbm
> FnZWQu
>
> Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm
> 5hYmxl
>
> LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2
> subWFu
>
> YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TW
> FuYWdl
>
> ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLm
> Nsb3Vk
>
> c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AE
> J0AA9j
>
> YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aE
> NvbnRl
>
> eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC
> 5jb250
>
> ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfg
> AOcQB-
>
> ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeG
> VjdXRv
>
> cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQ
> B-AAwC
>
> AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay
> 5qYXZh
>
> cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVG
> hyZWFk
>
> UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH
> 4AJXEA
>
> fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZW
> N1dG9y
>
> JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLl
> RocmVh
>
> ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW
> 9ucyRF
> bXB0eUxpc3R6uBe0PKee3gIAAHhweA
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Publish async job-4915 complete on message bus
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Wake up jobs related to job-4915
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Update db status for job-4915
>
> 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Wake up jobs joined with job-4915 and disjoin all subjobs created from
> job- 4915
>
> 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Done executing com.cloud.vm.VmWorkStart for job-4915
>
> 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Remove
> job-4915 from job monitoring
>
> 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> -U piotr -P
> Seneka22 chassis power status]: [Chassis Power is on
>
> ].
>
> 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> Cipher Suites
>
> Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
>
> Error response 0xc1 from Get PICMG Properities
>
> Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
>
> Invalid completion code received: Invalid command
>
> Discovered IPMB address 0x0
>
> ].
>
> 2021-11-17 14:47:27,805 DEBUG
> [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> (pool-2-thread-10:null)
> (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> successful and got the result [Chassis Power is on].
>
> 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> -U piotr -P
> Seneka22 chassis power status]: [Chassis Power is on
>
> ].
>
> 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> Cipher Suites
>
> Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
>
> Error response 0xc1 from Get PICMG Properities
>
> Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
>
> Invalid completion code received: Invalid command
>
> Discovered IPMB address 0x0
>
> ].
>
> 2021-11-17 14:47:27,806 DEBUG
> [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> (pool-2-thread-11:null)
> (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> successful and got the result [Chassis Power is on].
>
> 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> exception while executing
> org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> so(Con
> figDriveNetworkElement.java:597)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> data(C
> onfigDriveNetworkElement.java:225)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Elemen
> t(NetworkOrchestrator.java:1610)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Nic(Ne
> tworkOrchestrator.java:1936)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> (Netwo
> rkOrchestrator.java:1870)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:1201)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> ethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> egatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> xy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> anager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> ontext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> agedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> l(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> ithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> thCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> edCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> yncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> s.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Execut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> async job-4912, jobStatus: FAILED, resultCode: 530, result:
>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')"}
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> async
> job-4912 complete on message bus
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> jobs related to job-4912
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> status for job-4912
>
> 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> jobs joined with job-4912 and disjoin all subjobs created
>
>
Re: configdrive not working in 4.16
Posted by Wei ZHOU <us...@gmail.com>.
Hi Piotr,
Can you search HandleConfigDriveIsoCommand in agent.log and see if there
are any errors in when processing the command ?
-Wei
On Wed, 17 Nov 2021 at 16:25, Piotr Pisz <pi...@piszki.pl> wrote:
> Hi Wei,
>
> My KVM is CentOS 8.4 (upgrade from 4.15.2 without problems). VMs run on
> Ceph/RBD, NFS is from Ganesha.
>
> 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
> 2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Executing:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60
> 2021-11-17 16:20:20,075 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Executing while with timeout :
> 360000
> 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Execution is successful.
> 2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker]
> (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with
> KVMHAChecker
> [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
> -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h
> 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool:
> "nfs.piszki.lab"}].
> 2021-11-17 16:20:20,082 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) (logid:901784ad) Seq 1-4080261262397669854:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is
> beating...","wait":"0","bypassHostMaintenance":"false"}}] }
> 2021-11-17 16:20:23,687 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Request:Seq
> 1-4080261262397669855: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 16:20:23,688 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Processing command:
> com.cloud.agent.api.StopCommand
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find KVM connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: lxc:///
> 2021-11-17 16:20:23,688 INFO [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) No existing libvirtd
> connection found. Opening a new one
> 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find LXC connection
> for Instance: i-6-83-VM, continuing.
> 2021-11-17 16:20:23,689 WARN [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find a connection
> for Instance i-6-83-VM. Assuming the default connection.
> 2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd
> connection at: qemu:///system
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml:
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej
> do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> destroy_network_rules_for_vm --vmname i-6-83-VM
> 2021-11-17 16:20:23,691 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Executing while with timeout
> : 1800000
> 2021-11-17 16:20:23,823 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Execution is successful.
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) ebtables: No
> chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> ebtables: No chain/target/match by that name
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> iptables: No chain/target/match by that name.
> ipset v7.1: The set with the given name does not exist
>
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm :Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Try to stop the vm at first
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to stop VM :i-6-83-VM
> :
> org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny
> pasującej do nazwy „i-6-83-VM”
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.Connect.domainLookupByName(Unknown Source)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVMInternal(LibvirtComputingResource.java:3590)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3542)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:94)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:49)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1768)
> at com.cloud.agent.Agent.processRequest(Agent.java:661)
> at
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079)
> at com.cloud.utils.nio.Task.call(Task.java:83)
> at com.cloud.utils.nio.Task.call(Task.java:29)
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie
> odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
> 2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Can't get vm's status,
> assume it's dead already
> 2021-11-17 16:20:23,825 WARN [kvm.resource.LibvirtKvmAgentHook]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy script
> '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not
> available. Transformations will not be applied.
> 2021-11-17 16:20:23,826 WARN [kvm.resource.LibvirtKvmAgentHook]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy scripting engine is
> not initialized. Data transformation skipped.
> 2021-11-17 16:20:23,826 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) (logid:706e0dbf) Seq 1-4080261262397669855:
> { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2021-11-17 16:20:24,087 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) (logid:4f4315fe) Request:Seq
> 1-4080261262397669856: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1,
> Flags: 100011,
> [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}]
> }
>
> Regards,
> Piotr
>
>
> -----Original Message-----
> From: Wei ZHOU <us...@gmail.com>
> Sent: Wednesday, November 17, 2021 4:15 PM
> To: users <us...@cloudstack.apache.org>; ppisz@post.pl
> Subject: Re: configdrive not working in 4.16
>
> Hi Piotr,
>
> Any errors in agent.log if you use KVM ?
>
> -Wei
>
> On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <pp...@post.pl> wrote:
>
> > Hi,
> >
> > I have upgraded to 4.16 and am having trouble booting my VM on L2
> > network with ConfigDrive. I would have a request for help.
> >
> >
> >
> > Regards,
> >
> > Piotr
> >
> >
> >
> >
> >
> >
> >
> > 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We are allocating VM, increasing the used capacity of
> > this
> > host:1
> >
> > 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> > 3900
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> > 61916983296 ,Requested RAM: (4,00 GB) 4294967296
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We are allocating VM to the last host again, so
> > adjusting the reserved capacity if it is not less than required
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> >
> > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > (4,00
> > GB) 4294967296
> >
> > 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0,
> > old
> > reserved: 3900, actual total: 31200, total with overprovisioning:
> > 31200; new used:3900, reserved:0; requested
> > cpu:3900,alloc_from_last:true
> >
> > 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> > bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> > 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> > requested mem: (4,00
> > GB) 4294967296,alloc_from_last:true
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> > support requested CPU: 1 and requested speed: 3900
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested
> CPU:
> > 3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
> > 1.0
> >
> > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) We need to allocate to the last host again, so
> > checking if there is enough reserved capacity
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > (4,00
> > GB) 4294967296
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Host has enough CPU and RAM available
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> > 3900,
> > actual total: 31200, total with overprovisioning: 31200; requested
> > cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
> >
> > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> > reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> > requested
> > mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> > considerReservedCapacity?: true
> >
> > 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Network id=212 is already implemented
> >
> > 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > id=212
> >
> > 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > 1
> >
> > 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Asking ConfigDrive to prepare for
> > Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
> >
> > 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
> >
> > 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=service-offering, content=KVM-RBD-Custom
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=availability-zone, content=Piszki Lab
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=local-hostname, content=Brooklyn
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=local-ipv4, content=192.168.40.243
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-ipv4, content=192.168.40.243
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-hostname, content=null
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=public-keys, content=null
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=cloud-identifier,
> > content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
> >
> > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > filename=hypervisor-host-name, content=psc1.piszki.lab
> >
> > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing config drive creation command:
> > /usr/bin/genisoimage -o
> > /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> > -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r
> > -V
> > config-2 /tmp/configdrive12981467783887013707
> >
> > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> > /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> > -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> > config-2
> > /tmp/configdrive12981467783887013707
> >
> > 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Executing while with timeout : 300000
> >
> > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Execution is successful.
> >
> > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> > ISO-9660.
> >
> >
> >
> > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > drive/
> >
> > i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.Nf
> > sTO":{
> >
> > "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUns
> > upport
> >
> >
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> > "false"}}] }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfull
> > y
> > saved config drive at secondary
> > storage","wait":"0","bypassHostMaintenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> >
> > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Details from executing class
> > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> > config drive at secondary storage
> >
> > 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> > "83",
> > name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
> >
> >
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
> >
> > "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volu
> > mesToD
> >
> > isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> > "0","b
> > ypassHostMaintenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> > 10,
> >
> > [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypass
> > HostMa
> > intenance":"false"}}] }
> >
> > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> > StopAnswer } }
> >
> > 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> > id=212
> >
> > 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Changing active number of nics for network id=212 on
> > -1
> >
> > 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Asking ConfigDrive to release
> > NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-
> > null]
> >
> > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> > used=[1 TB], threshold=[89.99999761581421%].
> >
> > 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
> >
> > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> >
> > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> > drive/
> >
> >
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
> >
> > {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUn
> > suppor
> >
> > tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaint
> > enance
> > ":"false"}}] }
> >
> > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > Flags: 10,
> >
> > [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHost
> > Mainte
> > nance":"false"}}] }
> >
> > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
> > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> >
> > 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Unable to release some network resources for the VM
> > VM instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:661)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriv
> > eNetwo
> > rkElement.java:167)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > Nic(Ne
> > tworkOrchestrator.java:2238)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> > (Netwo
> > rkOrchestrator.java:2164)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachi
> > neMana
> > gerImpl.java:1836)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerIm
> > pl.jav
> > a:1826)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1373)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Successfully released storage resources for the VM VM
> > instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> > instance
> > {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > type="User"} in Starting state
> >
> > 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> > from [Starting] to [Stopped] with event [OperationFailed]. VM's original
> host:
> > Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> > "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> > null, host before state transition: Host {"id": "1", "name":
> > "psc1.piszki.lab",
> > "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
> >
> > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > applying
> > overprovisioning: 31200
> >
> > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and
> > RAM after applying overprovisioning: (61,66 GB) 66211950592
> >
> > 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> > actual total: 31200, total with overprovisioning: 31200; new used:
> > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> >
> > 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> > 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new
> used:
> > (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> > false,moveToReserveredfalse
> >
> > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Invocation exception, caused by:
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> > com.cloud.agent.api.Answer cannot be cast to class
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job
> origin:
> > 4912
> >
> > 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Unable to complete AsyncJobVO {id:4915, userId: 4, accountId: 4,
> > instanceType:
> > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> >
> > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAA
> > Zhdm9p
> >
> > ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdD
> > tMAAlj
> >
> > bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTm
> > FtZXQA
> >
> > EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubm
> > VycQB-
> >
> > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dG
> > lsL01h
> >
> > cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJW
> > drAgAE
> >
> > SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAA
> > AAAAAE
> >
> > AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAX
> > BzcgAO
> >
> > amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZX
> > KGrJUd
> >
> > C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcA
> > UH2sHD
> >
> > FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdA
> > AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> > cmdVersion: 0,
> > status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> > initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> > lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> > null}, job
> > origin:4912
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:597)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > data(C
> > onfigDriveNetworkElement.java:225)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Elemen
> > t(NetworkOrchestrator.java:1610)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Nic(Ne
> > tworkOrchestrator.java:1936)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > (Netwo
> > rkOrchestrator.java:1870)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1201)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
> >
> > rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaam
> > F2YS5s
> >
> > YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW
> > 9u0P0f
> >
> > Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0AB
> > VMamF2
> >
> > YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbm
> > c7WwAK
> >
> > c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZX
> > NzZWRF
> >
> > eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdW
> > QuYWdl
> >
> > bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbn
> > QuYXBp
> >
> > LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3
> > dlciBh
> >
> > bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcm
> > UgaW4g
> >
> > dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1
> > RyYWNl
> >
> > RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW
> > 1lbnRh
> >
> > CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH
> > 4ABUwA
> >
> > DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfg
> > AFTAAK
> >
> > bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2
> > NvbS5j
> >
> > bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbm
> > ZpZ0Ry
> >
> > aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AA
> > wBAAAA
> >
> > 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAA
> > ZKcQB-
> >
> > AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d2
> > 9ya09y
> >
> > Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW
> > 50cHBz
> >
> > cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQ
> > B-AA5x
> >
> > AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS
> > 5WaXJ0
> >
> > dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYX
> > ZhdAAQ
> >
> > b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3
> > EAfgAM
> >
> > Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbX
> > BsdAAd
> >
> > TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldA
> > AHMTEu
> >
> > MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAg
> > AAACtw
> >
> > dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbH
> > QAIURl
> >
> > bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADA
> > IAAAI2
> >
> > cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJX
> > EAfgAm
> >
> > c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveH
> > l0ABpW
> >
> > bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQ
> > AAF09x
> >
> > AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS
> > 5WbVdv
> >
> > cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icH
> > BzcQB-
> >
> > AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLm
> > ltcGwu
> >
> > QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcn
> > VuSW5D
> >
> > b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbm
> > FnZWQu
> >
> > Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm
> > 5hYmxl
> >
> > LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2
> > subWFu
> >
> > YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TW
> > FuYWdl
> >
> > ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLm
> > Nsb3Vk
> >
> > c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AE
> > J0AA9j
> >
> > YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aE
> > NvbnRl
> >
> > eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC
> > 5jb250
> >
> > ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfg
> > AOcQB-
> >
> > ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeG
> > VjdXRv
> >
> > cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQ
> > B-AAwC
> >
> > AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay
> > 5qYXZh
> >
> > cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVG
> > hyZWFk
> >
> > UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH
> > 4AJXEA
> >
> > fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZW
> > N1dG9y
> >
> > JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLl
> > RocmVh
> >
> > ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW
> > 9ucyRF
> > bXB0eUxpc3R6uBe0PKee3gIAAHhweA
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Publish async job-4915 complete on message bus
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Wake up jobs related to job-4915
> >
> > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Update db status for job-4915
> >
> > 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Wake up jobs joined with job-4915 and disjoin all subjobs created from
> > job- 4915
> >
> > 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Done executing com.cloud.vm.VmWorkStart for job-4915
> >
> > 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> > Remove
> > job-4915 from job monitoring
> >
> > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > -U piotr -P
> > Seneka22 chassis power status]: [Chassis Power is on
> >
> > ].
> >
> > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > Cipher Suites
> >
> > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> >
> > Error response 0xc1 from Get PICMG Properities
> >
> > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> >
> > Invalid completion code received: Invalid command
> >
> > Discovered IPMB address 0x0
> >
> > ].
> >
> > 2021-11-17 14:47:27,805 DEBUG
> > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > (pool-2-thread-10:null)
> > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> > successful and got the result [Chassis Power is on].
> >
> > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > -U piotr -P
> > Seneka22 chassis power status]: [Chassis Power is on
> >
> > ].
> >
> > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> > -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> > Cipher Suites
> >
> > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> >
> > Error response 0xc1 from Get PICMG Properities
> >
> > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> >
> > Invalid completion code received: Invalid command
> >
> > Discovered IPMB address 0x0
> >
> > ].
> >
> > 2021-11-17 14:47:27,806 DEBUG
> > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > (pool-2-thread-11:null)
> > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> > successful and got the result [Chassis Power is on].
> >
> > 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> > exception while executing
> > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
> >
> > java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> > be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> > so(Con
> > figDriveNetworkElement.java:597)
> >
> > at
> >
> > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> > data(C
> > onfigDriveNetworkElement.java:225)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Elemen
> > t(NetworkOrchestrator.java:1610)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > Nic(Ne
> > tworkOrchestrator.java:1936)
> >
> > at
> >
> > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> > (Netwo
> > rkOrchestrator.java:1870)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:1201)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> > Manage
> > rImpl.java:5802)
> >
> > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at
> >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodA
> > ccessorImpl.java:62)
> >
> > at
> >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatin
> > gMethodAccessorImpl.java:43)
> >
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >
> > at
> >
> > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> > xy.jav
> > a:107)
> >
> > at
> >
> > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> > anager
> > Impl.java:5967)
> >
> > at
> > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> > ontext
> > (AsyncJobManagerImpl.java:620)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> > agedCo
> > ntextRunnable.java:48)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> > l(Defa
> > ultManagedContext.java:55)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> > ithCon
> > text(DefaultManagedContext.java:102)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> > thCont
> > ext(DefaultManagedContext.java:52)
> >
> > at
> >
> > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> > edCont
> > extRunnable.java:45)
> >
> > at
> >
> > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> > yncJob
> > ManagerImpl.java:568)
> >
> > at
> >
> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> > s.java
> > :515)
> >
> > at
> > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Execut
> > or.java:1128)
> >
> > at
> >
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecu
> > tor.java:628)
> >
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> > async job-4912, jobStatus: FAILED, resultCode: 530, result:
> >
> > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> > [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> > cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > (com.cloud.agent.api.Answer and
> > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > of loader 'app')"}
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> > async
> > job-4912 complete on message bus
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > jobs related to job-4912
> >
> > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> > status for job-4912
> >
> > 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > jobs joined with job-4912 and disjoin all subjobs created
> >
> >
>
>
RE: configdrive not working in 4.16
Posted by Piotr Pisz <pi...@piszki.pl>.
Hi Wei,
My KVM is CentOS 8.4 (upgrade from 4.15.2 without problems). VMs run on Ceph/RBD, NFS is from Ganesha.
2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with KVMHAChecker for host IP [192.168.10.41] in pools [nfs.piszki.lab]
2021-11-17 16:20:20,074 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h 192.168.10.41 -r -t 60
2021-11-17 16:20:20,075 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Executing while with timeout : 360000
2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Execution is successful.
2021-11-17 16:20:20,082 DEBUG [kvm.resource.KVMHAChecker] (pool-1486-thread-1:null) (logid:901784ad) Checking heart beat with KVMHAChecker [{command="/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i nfs.piszki.lab -p /nfs -m /mnt/b2f9a49c-2efd-38e3-836c-c7e0ce9752e5 -h 192.168.10.41 -r -t 60 ", result: "null", log: "=====> ALIVE <=====", pool: "nfs.piszki.lab"}].
2021-11-17 16:20:20,082 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:901784ad) Seq 1-4080261262397669854: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is beating...","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 16:20:23,687 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:706e0dbf) Request:Seq 1-4080261262397669855: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToDisconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 16:20:23,688 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:706e0dbf) Processing command: com.cloud.agent.api.StopCommand
2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd connection at: qemu:///system
2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find KVM connection for Instance: i-6-83-VM, continuing.
2021-11-17 16:20:23,688 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd connection at: lxc:///
2021-11-17 16:20:23,688 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) No existing libvirtd connection found. Opening a new one
2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find LXC connection for Instance: i-6-83-VM, continuing.
2021-11-17 16:20:23,689 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Can not find a connection for Instance i-6-83-VM. Assuming the default connection.
2021-11-17 16:20:23,689 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:706e0dbf) Looking for libvirtd connection at: qemu:///system
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get dom xml: org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,690 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname i-6-83-VM
2021-11-17 16:20:23,691 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Executing while with timeout : 1800000
2021-11-17 16:20:23,823 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Execution is successful.
2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
ebtables: No chain/target/match by that name
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
ipset v7.1: The set with the given name does not exist
2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm :Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,824 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Try to stop the vm at first
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to stop VM :i-6-83-VM :
org.libvirt.LibvirtException: Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.Connect.domainLookupByName(Unknown Source)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVMInternal(LibvirtComputingResource.java:3590)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3542)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:94)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:49)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1768)
at com.cloud.agent.Agent.processRequest(Agent.java:661)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Failed to get vm status:Nie odnaleziono domeny: brak domeny pasującej do nazwy „i-6-83-VM”
2021-11-17 16:20:23,825 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:706e0dbf) Can't get vm's status, assume it's dead already
2021-11-17 16:20:23,825 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
2021-11-17 16:20:23,826 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-3:null) (logid:706e0dbf) Groovy scripting engine is not initialized. Data transformation skipped.
2021-11-17 16:20:23,826 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:706e0dbf) Seq 1-4080261262397669855: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 16:20:24,087 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:4f4315fe) Request:Seq 1-4080261262397669856: { Cmd , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"f4ce5698-9bed-3065-8442-ecac021b76e7-LibvirtComputingResource","privateNetwork":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"},"storageNetwork1":{"ip":"192.168.10.41","netmask":"255.255.255.0","mac":"be:3b:95:56:88:41","isSecurityGroupEnabled":"false"}},"wait":"20","bypassHostMaintenance":"false"}}] }
Regards,
Piotr
-----Original Message-----
From: Wei ZHOU <us...@gmail.com>
Sent: Wednesday, November 17, 2021 4:15 PM
To: users <us...@cloudstack.apache.org>; ppisz@post.pl
Subject: Re: configdrive not working in 4.16
Hi Piotr,
Any errors in agent.log if you use KVM ?
-Wei
On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <pp...@post.pl> wrote:
> Hi,
>
> I have upgraded to 4.16 and am having trouble booting my VM on L2
> network with ConfigDrive. I would have a request for help.
>
>
>
> Regards,
>
> Piotr
>
>
>
>
>
>
>
> 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We are allocating VM, increasing the used capacity of
> this
> host:1
>
> 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> 3900
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> 61916983296 ,Requested RAM: (4,00 GB) 4294967296
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We are allocating VM to the last host again, so
> adjusting the reserved capacity if it is not less than required
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> (4,00
> GB) 4294967296
>
> 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0,
> old
> reserved: 3900, actual total: 31200, total with overprovisioning:
> 31200; new used:3900, reserved:0; requested
> cpu:3900,alloc_from_last:true
>
> 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> requested mem: (4,00
> GB) 4294967296,alloc_from_last:true
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> support requested CPU: 1 and requested speed: 3900
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested CPU:
> 3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
> 1.0
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We need to allocate to the last host again, so
> checking if there is enough reserved capacity
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> (4,00
> GB) 4294967296
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Host has enough CPU and RAM available
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> 3900,
> actual total: 31200, total with overprovisioning: 31200; requested
> cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> requested
> mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> considerReservedCapacity?: true
>
> 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Network id=212 is already implemented
>
> 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> id=212
>
> 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Changing active number of nics for network id=212 on
> 1
>
> 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Asking ConfigDrive to prepare for
> Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
>
> 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
>
> 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=service-offering, content=KVM-RBD-Custom
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=availability-zone, content=Piszki Lab
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=local-hostname, content=Brooklyn
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=local-ipv4, content=192.168.40.243
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-ipv4, content=192.168.40.243
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-hostname, content=null
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-keys, content=null
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=cloud-identifier,
> content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=hypervisor-host-name, content=psc1.piszki.lab
>
> 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing config drive creation command:
> /usr/bin/genisoimage -o
> /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r
> -V
> config-2 /tmp/configdrive12981467783887013707
>
> 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> config-2
> /tmp/configdrive12981467783887013707
>
> 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing while with timeout : 300000
>
> 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Execution is successful.
>
> 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> ISO-9660.
>
>
>
> 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> drive/
>
> i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.Nf
> sTO":{
>
> "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUns
> upport
>
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> "false"}}] }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfull
> y
> saved config drive at secondary
> storage","wait":"0","bypassHostMaintenance":"false"}}] }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Details from executing class
> com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> config drive at secondary storage
>
> 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> "83",
> name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> type="User"} in Starting state
>
> 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
>
> "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volu
> mesToD
>
> isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> "0","b
> ypassHostMaintenance":"false"}}] }
>
> 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> 10,
>
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypass
> HostMa
> intenance":"false"}}] }
>
> 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> StopAnswer } }
>
> 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> id=212
>
> 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Changing active number of nics for network id=212 on
> -1
>
> 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Asking ConfigDrive to release
> NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-
> null]
>
> 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
>
> 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"config
> drive/
>
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
>
> {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUn
> suppor
>
> tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaint
> enance
> ":"false"}}] }
>
> 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> Flags: 10,
>
> [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHost
> Mainte
> nance":"false"}}] }
>
> 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
>
> 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Unable to release some network resources for the VM
> VM instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveI
> so(Con
> figDriveNetworkElement.java:661)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriv
> eNetwo
> rkElement.java:167)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> Nic(Ne
> tworkOrchestrator.java:2238)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release
> (Netwo
> rkOrchestrator.java:2164)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachi
> neMana
> gerImpl.java:1836)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerIm
> pl.jav
> a:1826)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:1373)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> ethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> egatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> xy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> anager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> ontext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> agedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> l(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> ithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> thCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> edCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> yncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> s.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Execut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Successfully released storage resources for the VM VM
> instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
>
> 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> instance
> {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> type="User"} in Starting state
>
> 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> from [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
> Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> null, host before state transition: Host {"id": "1", "name":
> "psc1.piszki.lab",
> "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
>
> 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and
> RAM after applying overprovisioning: (61,66 GB) 66211950592
>
> 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> actual total: 31200, total with overprovisioning: 31200; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>
> 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new used:
> (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> false,moveToReserveredfalse
>
> 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Invocation exception, caused by:
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin:
> 4912
>
> 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Unable to complete AsyncJobVO {id:4915, userId: 4, accountId: 4,
> instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAA
> Zhdm9p
>
> ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdD
> tMAAlj
>
> bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTm
> FtZXQA
>
> EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubm
> VycQB-
>
> AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dG
> lsL01h
>
> cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJW
> drAgAE
>
> SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAA
> AAAAAE
>
> AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAX
> BzcgAO
>
> amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZX
> KGrJUd
>
> C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcA
> UH2sHD
>
> FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdA
> AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0,
> status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> null}, job
> origin:4912
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> so(Con
> figDriveNetworkElement.java:597)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> data(C
> onfigDriveNetworkElement.java:225)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Elemen
> t(NetworkOrchestrator.java:1610)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Nic(Ne
> tworkOrchestrator.java:1936)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> (Netwo
> rkOrchestrator.java:1870)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:1201)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> ethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> egatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> xy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> anager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> ontext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> agedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> l(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> ithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> thCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> edCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> yncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> s.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Execut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaam
> F2YS5s
>
> YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW
> 9u0P0f
>
> Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0AB
> VMamF2
>
> YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbm
> c7WwAK
>
> c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZX
> NzZWRF
>
> eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdW
> QuYWdl
>
> bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbn
> QuYXBp
>
> LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3
> dlciBh
>
> bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcm
> UgaW4g
>
> dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1
> RyYWNl
>
> RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW
> 1lbnRh
>
> CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH
> 4ABUwA
>
> DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfg
> AFTAAK
>
> bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2
> NvbS5j
>
> bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbm
> ZpZ0Ry
>
> aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AA
> wBAAAA
>
> 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAA
> ZKcQB-
>
> AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d2
> 9ya09y
>
> Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW
> 50cHBz
>
> cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQ
> B-AA5x
>
> AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS
> 5WaXJ0
>
> dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYX
> ZhdAAQ
>
> b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3
> EAfgAM
>
> Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbX
> BsdAAd
>
> TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldA
> AHMTEu
>
> MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAg
> AAACtw
>
> dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbH
> QAIURl
>
> bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADA
> IAAAI2
>
> cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJX
> EAfgAm
>
> c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveH
> l0ABpW
>
> bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQ
> AAF09x
>
> AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS
> 5WbVdv
>
> cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icH
> BzcQB-
>
> AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLm
> ltcGwu
>
> QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcn
> VuSW5D
>
> b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbm
> FnZWQu
>
> Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm
> 5hYmxl
>
> LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2
> subWFu
>
> YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TW
> FuYWdl
>
> ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLm
> Nsb3Vk
>
> c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AE
> J0AA9j
>
> YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aE
> NvbnRl
>
> eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC
> 5jb250
>
> ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfg
> AOcQB-
>
> ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeG
> VjdXRv
>
> cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQ
> B-AAwC
>
> AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay
> 5qYXZh
>
> cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVG
> hyZWFk
>
> UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH
> 4AJXEA
>
> fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZW
> N1dG9y
>
> JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLl
> RocmVh
>
> ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW
> 9ucyRF
> bXB0eUxpc3R6uBe0PKee3gIAAHhweA
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Publish async job-4915 complete on message bus
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Wake up jobs related to job-4915
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Update db status for job-4915
>
> 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Wake up jobs joined with job-4915 and disjoin all subjobs created from
> job- 4915
>
> 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Done executing com.cloud.vm.VmWorkStart for job-4915
>
> 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Remove
> job-4915 from job monitoring
>
> 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> -U piotr -P
> Seneka22 chassis power status]: [Chassis Power is on
>
> ].
>
> 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623
> -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> Cipher Suites
>
> Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
>
> Error response 0xc1 from Get PICMG Properities
>
> Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
>
> Invalid completion code received: Invalid command
>
> Discovered IPMB address 0x0
>
> ].
>
> 2021-11-17 14:47:27,805 DEBUG
> [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> (pool-2-thread-10:null)
> (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> successful and got the result [Chassis Power is on].
>
> 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> -U piotr -P
> Seneka22 chassis power status]: [Chassis Power is on
>
> ].
>
> 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623
> -U piotr -P Seneka22 chassis power status]: [Unable to Get Channel
> Cipher Suites
>
> Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
>
> Error response 0xc1 from Get PICMG Properities
>
> Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
>
> Invalid completion code received: Invalid command
>
> Discovered IPMB address 0x0
>
> ].
>
> 2021-11-17 14:47:27,806 DEBUG
> [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> (pool-2-thread-11:null)
> (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> successful and got the result [Chassis Power is on].
>
> 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> exception while executing
> org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot
> be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveI
> so(Con
> figDriveNetworkElement.java:597)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUser
> data(C
> onfigDriveNetworkElement.java:225)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Elemen
> t(NetworkOrchestrator.java:1610)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> Nic(Ne
> tworkOrchestrator.java:1936)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare
> (Netwo
> rkOrchestrator.java:1870)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:1201)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachine
> Manage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> ethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> egatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerPro
> xy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineM
> anager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC
> ontext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man
> agedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal
> l(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW
> ithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi
> thCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag
> edCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As
> yncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
> s.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Execut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> async job-4912, jobStatus: FAILED, resultCode: 530, result:
>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> cannot be cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> of loader 'app')"}
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> async
> job-4912 complete on message bus
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> jobs related to job-4912
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> status for job-4912
>
> 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> jobs joined with job-4912 and disjoin all subjobs created
>
>
Re: configdrive not working in 4.16
Posted by Wei ZHOU <us...@gmail.com>.
Hi Piotr,
Any errors in agent.log if you use KVM ?
-Wei
On Wed, 17 Nov 2021 at 15:44, Piotr Pisz <pp...@post.pl> wrote:
> Hi,
>
> I have upgraded to 4.16 and am having trouble booting my VM on L2 network
> with ConfigDrive. I would have a request for help.
>
>
>
> Regards,
>
> Piotr
>
>
>
>
>
>
>
> 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We are allocating VM, increasing the used capacity of this
> host:1
>
> 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU: 3900
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> 61916983296 ,Requested RAM: (4,00 GB) 4294967296
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We are allocating VM to the last host again, so adjusting
> the reserved capacity if it is not less than required
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
>
> 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: (4,00
> GB) 4294967296
>
> 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0, old
> reserved: 3900, actual total: 31200, total with overprovisioning: 31200;
> new
> used:3900, reserved:0; requested cpu:3900,alloc_from_last:true
>
> 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
> bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> 66211950592;
> new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0; requested mem: (4,00
> GB) 4294967296,alloc_from_last:true
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to support
> requested CPU: 1 and requested speed: 3900
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Checking if host: 1 has enough capacity for requested CPU:
> 3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
> 1.0
>
> 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) We need to allocate to the last host again, so checking if
> there is enough reserved capacity
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: (4,00
> GB) 4294967296
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Host has enough CPU and RAM available
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> 3900,
> actual total: 31200, total with overprovisioning: 31200; requested
> cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
>
> 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
> reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592; requested
> mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> considerReservedCapacity?: true
>
> 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Network id=212 is already implemented
>
> 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> id=212
>
> 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Changing active number of nics for network id=212 on 1
>
> 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Asking ConfigDrive to prepare for
> Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
>
> 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
>
> 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=service-offering, content=KVM-RBD-Custom
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=availability-zone, content=Piszki Lab
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=local-hostname, content=Brooklyn
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=local-ipv4, content=192.168.40.243
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-ipv4, content=192.168.40.243
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-hostname, content=null
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=public-keys, content=null
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=cloud-identifier,
> content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
>
> 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> filename=hypervisor-host-name, content=psc1.piszki.lab
>
> 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing config drive creation command:
> /usr/bin/genisoimage -o /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> config-2 /tmp/configdrive12981467783887013707
>
> 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots -allow-lowercase
> -allow-multidot -cache-inodes -l -quiet -J -r -V config-2
> /tmp/configdrive12981467783887013707
>
> 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Executing while with timeout : 300000
>
> 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Execution is successful.
>
> 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Warning: creating filesystem that does not conform to
> ISO-9660.
>
>
>
> 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/
>
> i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{
>
> "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupport
>
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> "false"}}] }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfully
> saved config drive at secondary
> storage","wait":"0","bypassHostMaintenance":"false"}}] }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
>
> 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Details from executing class
> com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved config
> drive at secondary storage
>
> 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id: "83",
> name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> type="User"} in Starting state
>
> 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
>
> "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToD
>
> isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","b
> ypassHostMaintenance":"false"}}] }
>
> 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
>
> [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMa
> intenance":"false"}}] }
>
> 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, { StopAnswer
> }
> }
>
> 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Service SecurityGroup is not supported in the network
> id=212
>
> 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Changing active number of nics for network id=212 on -1
>
> 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Asking ConfigDrive to release
> NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-null]
>
> 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
> used=[1 TB], threshold=[89.99999761581421%].
>
> 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
>
> 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
>
> [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/
>
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
>
> {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsuppor
>
> tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance
> ":"false"}}] }
>
> 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, Flags: 10,
>
> [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHostMainte
> nance":"false"}}] }
>
> 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
> 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
>
> 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Unable to release some network resources for the VM VM
> instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIso(Con
> figDriveNetworkElement.java:661)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriveNetwo
> rkElement.java:167)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.releaseNic(Ne
> tworkOrchestrator.java:2238)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release(Netwo
> rkOrchestrator.java:2164)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachineMana
> gerImpl.java:1836)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerImpl.jav
> a:1826)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
> rImpl.java:1373)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Successfully released storage resources for the VM VM
> instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
>
> 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Successfully cleaned up resources for the VM VM instance
> {id: "83", name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> type="User"} in Starting state
>
> 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited from
> [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
> Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host: null,
> host before state transition: Host {"id": "1", "name": "psc1.piszki.lab",
> "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
>
> 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
> overprovisioning: 31200
>
> 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and RAM
> after applying overprovisioning: (61,66 GB) 66211950592
>
> 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
> actual total: 31200, total with overprovisioning: 31200; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>
> 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new used:
> (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> false,moveToReserveredfalse
>
> 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Invocation exception, caused by:
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
> com.cloud.agent.api.Answer cannot be cast to class
> com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
> and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin:
> 4912
>
> 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Unable
> to complete AsyncJobVO {id:4915, userId: 4, accountId: 4, instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
>
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9p
>
> ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAlj
>
> bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQA
>
> EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-
>
> AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01h
>
> cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAE
>
> SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAAE
>
> AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAO
>
> amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUd
>
> C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHD
>
> FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1Q
> YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0,
> status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed: null},
> job
> origin:4912
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveIso(Con
> figDriveNetworkElement.java:597)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUserdata(C
> onfigDriveNetworkElement.java:225)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElemen
> t(NetworkOrchestrator.java:1610)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(Ne
> tworkOrchestrator.java:1936)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(Netwo
> rkOrchestrator.java:1870)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
> rImpl.java:1201)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
>
> rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaamF2YS5s
>
> YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0f
>
> Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2
>
> YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAK
>
> c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRF
>
> eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdWQuYWdl
>
> bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbnQuYXBp
>
> LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3dlciBh
>
> bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcmUgaW4g
>
> dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNl
>
> RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRh
>
> CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH4ABUwA
>
> DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFTAAK
>
> bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2NvbS5j
>
> bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbmZpZ0Ry
>
> aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AAwBAAAA
>
> 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAAZKcQB-
>
> AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d29ya09y
>
> Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW50cHBz
>
> cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQB-AA5x
>
> AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS5WaXJ0
>
> dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQ
>
> b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3EAfgAM
>
> Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsdAAd
>
> TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldAAHMTEu
>
> MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAgAAACtw
>
> dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbHQAIURl
>
> bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADAIAAAI2
>
> cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJXEAfgAm
>
> c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpW
>
> bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQAAF09x
>
> AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS5WbVdv
>
> cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icHBzcQB-
>
> AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwu
>
> QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5D
>
> b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQu
>
> Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxl
>
> LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFu
>
> YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdl
>
> ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLmNsb3Vk
>
> c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AEJ0AA9j
>
> YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aENvbnRl
>
> eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250
>
> ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfgAOcQB-
>
> ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRv
>
> cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQB-AAwC
>
> AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZh
>
> cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFk
>
> UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH4AJXEA
>
> fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9y
>
> JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLlRocmVh
>
> ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW9ucyRF
> bXB0eUxpc3R6uBe0PKee3gIAAHhweA
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Publish async job-4915 complete on message bus
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Wake
> up jobs related to job-4915
>
> 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Update
> db status for job-4915
>
> 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Wake
> up jobs joined with job-4915 and disjoin all subjobs created from job- 4915
>
> 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Done
> executing com.cloud.vm.VmWorkStart for job-4915
>
> 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
> Remove
> job-4915 from job monitoring
>
> 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> command
> [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623 -U piotr -P
> Seneka22 chassis power status]: [Chassis Power is on
>
> ].
>
> 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623 -U
> piotr -P Seneka22 chassis power status]: [Unable to Get Channel Cipher
> Suites
>
> Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
>
> Error response 0xc1 from Get PICMG Properities
>
> Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
>
> Invalid completion code received: Invalid command
>
> Discovered IPMB address 0x0
>
> ].
>
> 2021-11-17 14:47:27,805 DEBUG
> [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] (pool-2-thread-10:null)
> (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> successful and got the result [Chassis Power is on].
>
> 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> command
> [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623 -U piotr -P
> Seneka22 chassis power status]: [Chassis Power is on
>
> ].
>
> 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
> command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623 -U
> piotr -P Seneka22 chassis power status]: [Unable to Get Channel Cipher
> Suites
>
> Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
>
> Error response 0xc1 from Get PICMG Properities
>
> Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
>
> Invalid completion code received: Invalid command
>
> Discovered IPMB address 0x0
>
> ].
>
> 2021-11-17 14:47:27,806 DEBUG
> [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] (pool-2-thread-11:null)
> (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> successful and got the result [Chassis Power is on].
>
> 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
> exception while executing
> org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
>
> java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
> cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveIso(Con
> figDriveNetworkElement.java:597)
>
> at
>
> com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUserdata(C
> onfigDriveNetworkElement.java:225)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElemen
> t(NetworkOrchestrator.java:1610)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(Ne
> tworkOrchestrator.java:1936)
>
> at
>
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(Netwo
> rkOrchestrator.java:1870)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
> rImpl.java:1201)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
> rImpl.java:5802)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
> ccessorImpl.java:62)
>
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
> gMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
>
> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
> a:107)
>
> at
>
> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
> Impl.java:5967)
>
> at
> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
> (AsyncJobManagerImpl.java:620)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
> ntextRunnable.java:48)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
> ultManagedContext.java:55)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
> text(DefaultManagedContext.java:102)
>
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
> ext(DefaultManagedContext.java:52)
>
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
> extRunnable.java:45)
>
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
> ManagerImpl.java:568)
>
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
> :515)
>
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
> or.java:1128)
>
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
> tor.java:628)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete async
> job-4912, jobStatus: FAILED, resultCode: 530, result:
>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"er
> rorcode":"530","errortext":"class com.cloud.agent.api.Answer cannot be cast
> to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
> (com.cloud.agent.api.Answer and
> com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
> loader 'app')"}
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish async
> job-4912 complete on message bus
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up jobs
> related to job-4912
>
> 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db
> status
> for job-4912
>
> 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up jobs
> joined with job-4912 and disjoin all subjobs created
>
>
RE: configdrive not working in 4.16
Posted by Piotr Pisz <pi...@piszki.pl>.
Hi,
On 4.15, this behavior occurred when Primary Storage was overflowing to the
size of the 'disabled threshold'. Here the size of the indicator does not
matter.
Asking ConfigDrive to release
NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-null]
Verifying image storage [1]. Capacity: total=[9 TB],used=[1 TB],
threshold=[89.99999761581421%].
Deleting config drive ISO for vm: i-6-83-VM
Regards,
Piotr
-----Original Message-----
From: Piotr Pisz <pp...@post.pl>
Sent: Wednesday, November 17, 2021 2:55 PM
To: users@cloudstack.apache.org
Subject: configdrive not working in 4.16
Hi,
I have upgraded to 4.16 and am having trouble booting my VM on L2 network
with ConfigDrive. I would have a request for help.
Regards,
Piotr
2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
overprovisioning: 31200
2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) We are allocating VM, increasing the used capacity of this
host:1
2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU: 3900
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
61916983296 ,Requested RAM: (4,00 GB) 4294967296
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) We are allocating VM to the last host again, so adjusting
the reserved capacity if it is not less than required
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: (4,00
GB) 4294967296
2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: 0, old
reserved: 3900, actual total: 31200, total with overprovisioning: 31200; new
used:3900, reserved:0; requested cpu:3900,alloc_from_last:true
2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: (0
bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0; requested mem: (4,00
GB) 4294967296,alloc_from_last:true
2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to support
requested CPU: 1 and requested speed: 3900
2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Checking if host: 1 has enough capacity for requested CPU:
3900 and requested RAM: (4,00 GB) 4294967296 , cpuOverprovisioningFactor:
1.0
2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
overprovisioning: 31200
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) We need to allocate to the last host again, so checking if
there is enough reserved capacity
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: (4,00
GB) 4294967296
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Host has enough CPU and RAM available
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved: 3900,
actual total: 31200, total with overprovisioning: 31200; requested
cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) 0,
reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592; requested
mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
considerReservedCapacity?: true
2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Network id=212 is already implemented
2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Service SecurityGroup is not supported in the network
id=212
2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Changing active number of nics for network id=212 on 1
2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Asking ConfigDrive to prepare for
Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=service-offering, content=KVM-RBD-Custom
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=availability-zone, content=Piszki Lab
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=local-hostname, content=Brooklyn
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=local-ipv4, content=192.168.40.243
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=public-ipv4, content=192.168.40.243
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=public-hostname, content=null
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=public-keys, content=null
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=cloud-identifier,
content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
filename=hypervisor-host-name, content=psc1.piszki.lab
2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Executing config drive creation command:
/usr/bin/genisoimage -o /tmp/configdrive12981467783887013707/i-6-83-VM.iso
-ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
config-2 /tmp/configdrive12981467783887013707
2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
/tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots -allow-lowercase
-allow-multidot -cache-inodes -l -quiet -J -r -V config-2
/tmp/configdrive12981467783887013707
2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Executing while with timeout : 300000
2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Execution is successful.
2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Warning: creating filesystem that does not conform to
ISO-9660.
2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/
i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.NfsTO":{
"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsupport
edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
"false"}}] }
2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfully
saved config drive at secondary
storage","wait":"0","bypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Details from executing class
com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved config
drive at secondary storage
2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Cleaning up resources for the vm VM instance {id: "83",
name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
type="User"} in Starting state
2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
"false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"volumesToD
isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":"0","b
ypassHostMaintenance":"false"}}] }
2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMa
intenance":"false"}}] }
2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, { StopAnswer }
}
2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Service SecurityGroup is not supported in the network
id=212
2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Changing active number of nics for network id=212 on -1
2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Asking ConfigDrive to release
NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243-null]
2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 TB],
used=[1 TB], threshold=[89.99999761581421%].
2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/
i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
{"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnUnsuppor
tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMaintenance
":"false"}}] }
2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHostMainte
nance":"false"}}] }
2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , MgmtId:
260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Unable to release some network resources for the VM VM
instance {id: "83", name: "i-6-83-VM", uuid:
"6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
at
com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIso(Con
figDriveNetworkElement.java:661)
at
com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDriveNetwo
rkElement.java:167)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.releaseNic(Ne
tworkOrchestrator.java:2238)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release(Netwo
rkOrchestrator.java:2164)
at
com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMachineMana
gerImpl.java:1836)
at
com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerImpl.jav
a:1826)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:1373)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:5802)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
ccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
gMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
a:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
Impl.java:5967)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
ntextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
ultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
text(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
ext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
extRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
ManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Successfully released storage resources for the VM VM
instance {id: "83", name: "i-6-83-VM", uuid:
"6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting state
2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Successfully cleaned up resources for the VM VM instance
{id: "83", name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
type="User"} in Starting state
2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
"6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited from
[Starting] to [Stopped] with event [OperationFailed]. VM's original host:
Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
"173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host: null,
host before state transition: Host {"id": "1", "name": "psc1.piszki.lab",
"uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after applying
overprovisioning: 31200
2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 and RAM
after applying overprovisioning: (61,66 GB) 66211950592
2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: 0,
actual total: 31200, total with overprovisioning: 31200; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new used:
(0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
false,moveToReserveredfalse
2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Invocation exception, caused by:
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
(logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: class
com.cloud.agent.api.Answer cannot be cast to class
com.cloud.agent.api.HandleConfigDriveIsoAnswer (com.cloud.agent.api.Answer
and com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin:
4912
2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Unable
to complete AsyncJobVO {id:4915, userId: 4, accountId: 4, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9p
ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAlj
bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQA
EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-
AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01h
cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAE
SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAAE
AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAO
amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUd
C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHD
FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1Q
YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0,
status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed: null}, job
origin:4912
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
at
com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveIso(Con
figDriveNetworkElement.java:597)
at
com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUserdata(C
onfigDriveNetworkElement.java:225)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElemen
t(NetworkOrchestrator.java:1610)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(Ne
tworkOrchestrator.java:1936)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(Netwo
rkOrchestrator.java:1870)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:1201)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:5802)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
ccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
gMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
a:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
Impl.java:5967)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
ntextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
ultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
text(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
ext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
extRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
ManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
Complete async job-4915, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAaamF2YS5s
YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0f
Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2
YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAK
c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRF
eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xvdWQuYWdl
bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdlbnQuYXBp
LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFuc3dlciBh
bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBhcmUgaW4g
dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNl
RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRh
CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH4ABUwA
DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFTAAK
bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQAM2NvbS5j
bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNvbmZpZ0Ry
aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-AAwBAAAA
4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEAAAZKcQB-
AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0d29ya09y
Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVtZW50cHBz
cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdOcQB-AA5x
AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52bS5WaXJ0
dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQ
b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bwc3EAfgAM
Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsdAAd
TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNldAAHMTEu
MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAMAgAAACtw
dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbHQAIURl
bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4ADAIAAAI2
cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4AJXEAfgAm
c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpW
bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQAAF09x
AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS5WbVdv
cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icHBzcQB-
AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwu
QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5D
b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQu
Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxl
LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFu
YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdl
ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLmNsb3Vk
c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AEJ0AA9j
YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0aENvbnRl
eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250
ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEAfgAOcQB-
ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRv
cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZzcQB-AAwC
AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZh
cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFk
UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH4AJXEA
fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9y
JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLlRocmVh
ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0aW9ucyRF
bXB0eUxpc3R6uBe0PKee3gIAAHhweA
2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2)
Publish async job-4915 complete on message bus
2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Wake
up jobs related to job-4915
2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Update
db status for job-4915
2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Wake
up jobs joined with job-4915 and disjoin all subjobs created from job- 4915
2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Done
executing com.cloud.vm.VmWorkStart for job-4915
2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) (logid:b59e4ee2) Remove
job-4915 from job monitoring
2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for command
[/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623 -U piotr -P
Seneka22 chassis power status]: [Chassis Power is on
].
2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-10:null) (logid:da9a8ffa) Process standard error output
command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p 623 -U
piotr -P Seneka22 chassis power status]: [Unable to Get Channel Cipher
Suites
Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
Error response 0xc1 from Get PICMG Properities
Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
Invalid completion code received: Invalid command
Discovered IPMB address 0x0
].
2021-11-17 14:47:27,805 DEBUG
[o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] (pool-2-thread-10:null)
(logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
successful and got the result [Chassis Power is on].
2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for command
[/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623 -U piotr -P
Seneka22 chassis power status]: [Chassis Power is on
].
2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
(pool-2-thread-11:null) (logid:da9a8ffa) Process standard error output
command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p 623 -U
piotr -P Seneka22 chassis power status]: [Unable to Get Channel Cipher
Suites
Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
Error response 0xc1 from Get PICMG Properities
Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
Invalid completion code received: Invalid command
Discovered IPMB address 0x0
].
2021-11-17 14:47:27,806 DEBUG
[o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] (pool-2-thread-11:null)
(logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
successful and got the result [Chassis Power is on].
2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Unexpected
exception while executing
org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be
cast to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')
at
com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriveIso(Con
figDriveNetworkElement.java:597)
at
com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUserdata(C
onfigDriveNetworkElement.java:225)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElemen
t(NetworkOrchestrator.java:1610)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(Ne
tworkOrchestrator.java:1936)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(Netwo
rkOrchestrator.java:1870)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:1201)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage
rImpl.java:5802)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
ccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
gMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav
a:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager
Impl.java:5967)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext
(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo
ntextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa
ultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon
text(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont
ext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont
extRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob
ManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete async
job-4912, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"er
rorcode":"530","errortext":"class com.cloud.agent.api.Answer cannot be cast
to class com.cloud.agent.api.HandleConfigDriveIsoAnswer
(com.cloud.agent.api.Answer and
com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module of
loader 'app')"}
2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish async
job-4912 complete on message bus
2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up jobs
related to job-4912
2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update db status
for job-4912
2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up jobs
joined with job-4912 and disjoin all subjobs created