You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Daan Hoogland <da...@gmail.com> on 2021/12/13 09:19:10 UTC
Re: configdrive not working in 4.16
I think `this QEMU binary` is the key phrase here, Piotr.
On Tue, Nov 23, 2021 at 9:51 AM Piotr Pisz <pi...@piszki.pl> wrote:
> Hi Wei,
>
> Unfortunately, neither systemvm, nor any other vm, is starting due to the
> message: libvirt.LibvirtException: unsupported configuration: io uring is
> not supported by this QEMU binary
>
> In one LAB I have a custom kernel: 5.4.129-1.el8.elrepo.x86_64 and
> libvirtd (libvirt) 7.6.0 with qemu 6.0.0 (CentOS 8.5 with advanced
> virtualization repo)
>
> But in the second, it comes with the system: 4.18.0-240.22.1.el8_3.x86_64
> and libvirtd (libvirt) 7.0.0 with qemu-kvm-5.2.0-16.el8 (CentOS 8.2), error
> is in both
>
> IO uring works on host:
>
> root@psc1 /h/piotr# grep io_uring_setup /proc/kallsyms
> ffffffffa274a090 t io_uring_setup
> ffffffffa274a150 T __x64_sys_io_uring_setup
> ffffffffa274a170 T __ia32_sys_io_uring_setup
> ffffffffa38ff720 d event_exit__io_uring_setup
> ffffffffa38ff7c0 d event_enter__io_uring_setup
> ffffffffa38ff860 d __syscall_meta__io_uring_setup
> ffffffffa38ff8a0 d args__io_uring_setup
> ffffffffa38ff8b0 d types__io_uring_setup
> ffffffffa3fd8658 t __event_exit__io_uring_setup
> ffffffffa3fd8660 t __event_enter__io_uring_setup
> ffffffffa3fda4b0 t __p_syscall_meta__io_uring_setup
> ffffffffa3fddd20 t _eil_addr___ia32_sys_io_uring_setup
> ffffffffa3fddd30 t _eil_addr___x64_sys_io_uring_setup
>
> I don't know what to do with it yet.
>
> Regards,
> Piotr
>
>
>
> -----Original Message-----
> From: Wei ZHOU <us...@gmail.com>
> Sent: Tuesday, November 23, 2021 9:13 AM
> To: users <us...@cloudstack.apache.org>; piotr@piszki.pl
> Subject: Re: configdrive not working in 4.16
>
> Hi Piotr,
>
> Could you please check if all your management servers, cloudstack agents,
> and system vms (SSVM/CPVM) are running with 4.16.0.0 ?
> It's mostly like your SSVM is not 4.16.0.0.
> When processing HandleConfigDriveIsoCommand, SSVM with the old version
> returns Answer (not HandleConfigDriveIsoAnswer).
>
> -Wei
>
>
> On Mon, 22 Nov 2021 at 10:25, Piotr Pisz <pi...@piszki.pl> wrote:
>
> > Hi Daan,
> >
> > Unfortunately, I couldn't deal with this problem and withdrawn the
> changes.
> > I have two LAB environments, both behaved the same. In one, MGMT is
> > installed on Ubuntu 20, in the other, on CentOS 8, in both cases the
> > KVM is CentOS 8.4/8.5
> >
> > On CentOS 8:
> > openjdk version "11.0.13" 2021-10-19 LTS OpenJDK Runtime Environment
> > 18.9 (build 11.0.13+8-LTS) OpenJDK 64-Bit Server VM 18.9 (build
> > 11.0.13+8-LTS, mixed mode, sharing)
> >
> > Packages downloaded from official page.
> >
> > I also found a problem with tags, after the upgrade I was not able to
> > add new tags with UI, old tags (added with legacy client) in primary /
> > secondary storage were not displayed correctly.
> > Right now I am building a LAB with version 4.16, I need a clean DB to
> > compare.
> >
> > Regards,
> > Piotr
> >
> >
> > -----Original Message-----
> > From: Daan Hoogland <da...@gmail.com>
> > Sent: Monday, November 22, 2021 9:17 AM
> > To: users <us...@cloudstack.apache.org>; piotr@piszki.pl
> > Subject: Re: configdrive not working in 4.16
> >
> > Piotr,
> > have you gotten anywhere with this?
> > the IPMI exception seems unrelated,
> > and the class cast exceptions seem to me to be a java version issue?
> > can you give the versions of relevant platform version? (at least os,
> > jvm) Have you built yourself or downloaded any package?
> >
> > On Wed, Nov 17, 2021 at 3:27 PM Piotr Pisz <pi...@piszki.pl> wrote:
> >
> > >
> > >
> > > Hi,
> > >
> > > I have upgraded to 4.16 and am having trouble booting my VM on L2
> > > network with ConfigDrive. I would have a request for help.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Piotr
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > > applying
> > > overprovisioning: 31200
> > >
> > > 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) We are allocating VM, increasing the used capacity
> > > of this
> > > host:1
> > >
> > > 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU:
> > > 3900
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB)
> > > 61916983296 ,Requested RAM: (4,00 GB) 4294967296
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) We are allocating VM to the last host again, so
> > > adjusting the reserved capacity if it is not less than required
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> > >
> > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > > (4,00
> > > GB) 4294967296
> > >
> > > 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used:
> > > 0, old
> > > reserved: 3900, actual total: 31200, total with overprovisioning:
> > > 31200; new used:3900, reserved:0; requested
> > > cpu:3900,alloc_from_last:true
> > >
> > > 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used:
> > > (0
> > > bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB)
> > > 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0;
> > > requested mem: (4,00
> > > GB) 4294967296,alloc_from_last:true
> > >
> > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to
> > > support requested CPU: 1 and requested speed: 3900
> > >
> > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Checking if host: 1 has enough capacity for
> > > requested
> > CPU:
> > > 3900 and requested RAM: (4,00 GB) 4294967296 ,
> cpuOverprovisioningFactor:
> > > 1.0
> > >
> > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > > applying
> > > overprovisioning: 31200
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) We need to allocate to the last host again, so
> > > checking if there is enough reserved capacity
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM:
> > > (4,00
> > > GB) 4294967296
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Host has enough CPU and RAM available
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved:
> > > 3900,
> > > actual total: 31200, total with overprovisioning: 31200; requested
> > > cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true
> > >
> > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes)
> > > 0,
> > > reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592;
> > > requested
> > > mem: (4,00 GB) 4294967296, alloc_from_last_host?: true ,
> > > considerReservedCapacity?: true
> > >
> > > 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Network id=212 is already implemented
> > >
> > > 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Service SecurityGroup is not supported in the
> > > network
> > > id=212
> > >
> > > 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Changing active number of nics for network id=212
> > > on
> > > 1
> > >
> > > 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Asking ConfigDrive to prepare for
> > > Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243]
> > >
> > > 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9
> > > TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9
> > > TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM
> > >
> > > 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=service-offering, content=KVM-RBD-Custom
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=availability-zone, content=Piszki Lab
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=local-hostname, content=Brooklyn
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=local-ipv4, content=192.168.40.243
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=public-ipv4, content=192.168.40.243
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=public-hostname, content=null
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=public-keys, content=null
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=cloud-identifier,
> > > content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26}
> > >
> > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata,
> > > filename=hypervisor-host-name, content=psc1.piszki.lab
> > >
> > > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Executing config drive creation command:
> > > /usr/bin/genisoimage -o
> > > /tmp/configdrive12981467783887013707/i-6-83-VM.iso
> > > -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J
> > > -r -V
> > > config-2 /tmp/configdrive12981467783887013707
> > >
> > > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o
> > > /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots
> > > -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V
> > > config-2
> > > /tmp/configdrive12981467783887013707
> > >
> > > 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Executing while with timeout : 300000
> > >
> > > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Execution is successful.
> > >
> > > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Warning: creating filesystem that does not conform
> > > to ISO-9660.
> > >
> > >
> > >
> > > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> > >
> > > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"conf
> > > ig
> > > drive/
> > >
> > > i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to.
> > > Nf
> > > sTO":{
> > >
> > > "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnU
> > > ns
> > > upport
> > >
> > >
> >
> edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance":
> > > "false"}}] }
> > >
> > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146:
> > > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > > Flags: 10,
> > > [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfu
> > > ll
> > > y
> > > saved config drive at secondary
> > > storage","wait":"0","bypassHostMaintenance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: ,
> MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> > >
> > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Details from executing class
> > > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved
> > > config drive at secondary storage
> > >
> > > 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id:
> > > "83",
> > > name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > > type="User"} in Starting state
> > >
> > > 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId:
> > > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011,
> > >
> > >
> >
> [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":
> > >
> > > "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"vo
> > > lu
> > > mesToD
> > >
> > > isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait":
> > > "0","b
> > > ypassHostMaintenance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545:
> > > Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags:
> > > 10,
> > >
> > > [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypa
> > > ss
> > > HostMa
> > > intenance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId:
> > > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, {
> > > StopAnswer } }
> > >
> > > 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Service SecurityGroup is not supported in the
> > > network
> > > id=212
> > >
> > > 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Changing active number of nics for network id=212
> > > on
> > > -1
> > >
> > > 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Asking ConfigDrive to release
> > > NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.24
> > > 3-
> > > null]
> > >
> > > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9
> > > TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9
> > > TB],
> > > used=[1 TB], threshold=[89.99999761581421%].
> > >
> > > 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM
> > >
> > > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011,
> > >
> > > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"conf
> > > ig
> > > drive/
> > >
> > >
> >
> i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO":
> > >
> > > {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOn
> > > Un
> > > suppor
> > >
> > > tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMai
> > > nt
> > > enance
> > > ":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147:
> > > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1,
> > > Flags: 10,
> > >
> > > [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHo
> > > st
> > > Mainte
> > > nance":"false"}}] }
> > >
> > > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: ,
> MgmtId:
> > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } }
> > >
> > > 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Unable to release some network resources for the VM
> > > VM instance {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting
> > > state
> > >
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer
> > > cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriv
> > > eI
> > > so(Con
> > > figDriveNetworkElement.java:661)
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDr
> > > iv
> > > eNetwo
> > > rkElement.java:167)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.relea
> > > se
> > > Nic(Ne
> > > tworkOrchestrator.java:2238)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.relea
> > > se
> > > (Netwo
> > > rkOrchestrator.java:2164)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMac
> > > hi
> > > neMana
> > > gerImpl.java:1836)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManager
> > > Im
> > > pl.jav
> > > a:1826)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi
> > > ne
> > > Manage
> > > rImpl.java:1373)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi
> > > ne
> > > Manage
> > > rImpl.java:5802)
> > >
> > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati
> > > ve
> > > Method)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ
> > > eM
> > > ethodA
> > > ccessorImpl.java:62)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D
> > > el
> > > egatin
> > > gMethodAccessorImpl.java:43)
> > >
> > > at
> > > java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >
> > > at
> > >
> > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerP
> > > ro
> > > xy.jav
> > > a:107)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachin
> > > eM
> > > anager
> > > Impl.java:5967)
> > >
> > > at
> > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102
> > > )
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runI
> > > nC
> > > ontext
> > > (AsyncJobManagerImpl.java:620)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M
> > > an
> > > agedCo
> > > ntextRunnable.java:48)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c
> > > al
> > > l(Defa
> > > ultManagedContext.java:55)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal
> > > lW
> > > ithCon
> > > text(DefaultManagedContext.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run
> > > Wi
> > > thCont
> > > ext(DefaultManagedContext.java:52)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man
> > > ag
> > > edCont
> > > extRunnable.java:45)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(
> > > As
> > > yncJob
> > > ManagerImpl.java:568)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Execut
> > > or
> > > s.java
> > > :515)
> > >
> > > at
> > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPo
> > > ol
> > > Execut
> > > or.java:1128)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP
> > > oo
> > > lExecu
> > > tor.java:628)
> > >
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Successfully released storage resources for the VM
> > > VM instance {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting
> > > state
> > >
> > > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Successfully cleaned up resources for the VM VM
> > > instance
> > > {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb",
> > > type="User"} in Starting state
> > >
> > > 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid:
> > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited
> > > from [Starting] to [Stopped] with event [OperationFailed]. VM's
> > > original
> > host:
> > > Host {"id": "1", "name": "psc1.piszki.lab", "uuid":
> > > "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host:
> > > null, host before state transition: Host {"id": "1", "name":
> > > "psc1.piszki.lab",
> > > "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}
> > >
> > > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after
> > > applying
> > > overprovisioning: 31200
> > >
> > > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592
> > > and RAM after applying overprovisioning: (61,66 GB) 66211950592
> > >
> > > 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved:
> > > 0, actual total: 31200, total with overprovisioning: 31200; new used:
> > > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> > >
> > > 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB)
> > > 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new
> > used:
> > > (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved:
> > > false,moveToReserveredfalse
> > >
> > > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Invocation exception, caused by:
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer
> > > cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663)
> > > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException:
> > > class com.cloud.agent.api.Answer cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Done with run of VM work job:
> > > com.cloud.vm.VmWorkStart for VM 83, job
> > origin:
> > > 4912
> > >
> > > 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Unable to complete AsyncJobVO {id:4915, userId: 4,
> > > accountId: 4,
> > > instanceType:
> > > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> > >
> > > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRM
> > > AA
> > > Zhdm9p
> > >
> > > ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlz
> > > dD
> > > tMAAlj
> > >
> > > bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFs
> > > Tm
> > > FtZXQA
> > >
> > > EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFu
> > > bm
> > > VycQB-
> > >
> > > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91
> > > dG
> > > lsL01h
> > >
> > > cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-Ztlbw
> > > JW
> > > drAgAE
> > >
> > > SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAA
> > > AA
> > > AAAAAE
> > >
> > > AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAA
> > > AX
> > > BzcgAO
> > >
> > > amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1i
> > > ZX
> > > KGrJUd
> > >
> > > C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1h
> > > cA
> > > UH2sHD
> > >
> > > FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAAB
> > > dA AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> > > cmdVersion: 0,
> > > status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null,
> > > initMsid: 260497154630684, completeMsid: null, lastUpdated: null,
> > > lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed:
> > > null}, job
> > > origin:4912
> > >
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer
> > > cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriv
> > > eI
> > > so(Con
> > > figDriveNetworkElement.java:597)
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUs
> > > er
> > > data(C
> > > onfigDriveNetworkElement.java:225)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa
> > > re
> > > Elemen
> > > t(NetworkOrchestrator.java:1610)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa
> > > re
> > > Nic(Ne
> > > tworkOrchestrator.java:1936)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa
> > > re
> > > (Netwo
> > > rkOrchestrator.java:1870)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi
> > > ne
> > > Manage
> > > rImpl.java:1201)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi
> > > ne
> > > Manage
> > > rImpl.java:5802)
> > >
> > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati
> > > ve
> > > Method)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ
> > > eM
> > > ethodA
> > > ccessorImpl.java:62)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D
> > > el
> > > egatin
> > > gMethodAccessorImpl.java:43)
> > >
> > > at
> > > java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >
> > > at
> > >
> > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerP
> > > ro
> > > xy.jav
> > > a:107)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachin
> > > eM
> > > anager
> > > Impl.java:5967)
> > >
> > > at
> > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102
> > > )
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runI
> > > nC
> > > ontext
> > > (AsyncJobManagerImpl.java:620)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M
> > > an
> > > agedCo
> > > ntextRunnable.java:48)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c
> > > al
> > > l(Defa
> > > ultManagedContext.java:55)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal
> > > lW
> > > ithCon
> > > text(DefaultManagedContext.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run
> > > Wi
> > > thCont
> > > ext(DefaultManagedContext.java:52)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man
> > > ag
> > > edCont
> > > extRunnable.java:45)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(
> > > As
> > > yncJob
> > > ManagerImpl.java:568)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Execut
> > > or
> > > s.java
> > > :515)
> > >
> > > at
> > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPo
> > > ol
> > > Execut
> > > or.java:1128)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP
> > > oo
> > > lExecu
> > > tor.java:628)
> > >
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Complete async job-4915, jobStatus: FAILED,
> resultCode: 0, result:
> > >
> > > rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAa
> > > am
> > > F2YS5s
> > >
> > > YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0
> > > aW
> > > 9u0P0f
> > >
> > > Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0
> > > AB
> > > VMamF2
> > >
> > > YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJp
> > > bm
> > > c7WwAK
> > >
> > > c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHBy
> > > ZX
> > > NzZWRF
> > >
> > > eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xv
> > > dW
> > > QuYWdl
> > >
> > > bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdl
> > > bn
> > > QuYXBp
> > >
> > > LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFu
> > > c3
> > > dlciBh
> > >
> > > bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBh
> > > cm
> > > UgaW4g
> > >
> > > dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFj
> > > a1
> > > RyYWNl
> > >
> > > RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVs
> > > ZW
> > > 1lbnRh
> > >
> > > CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVx
> > > AH
> > > 4ABUwA
> > >
> > > DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEA
> > > fg
> > > AFTAAK
> > >
> > > bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQA
> > > M2
> > > NvbS5j
> > >
> > > bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNv
> > > bm
> > > ZpZ0Ry
> > >
> > > aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB-
> > > AA
> > > wBAAAA
> > >
> > > 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEA
> > > AA
> > > ZKcQB-
> > >
> > > AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0
> > > d2
> > > 9ya09y
> > >
> > > Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVt
> > > ZW
> > > 50cHBz
> > >
> > > cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdO
> > > cQ
> > > B-AA5x
> > >
> > > AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52
> > > bS
> > > 5WaXJ0
> > >
> > > dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5q
> > > YX
> > > ZhdAAQ
> > >
> > > b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bw
> > > c3
> > > EAfgAM
> > >
> > > Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJ
> > > bX
> > > BsdAAd
> > >
> > > TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNl
> > > dA
> > > AHMTEu
> > >
> > > MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAM
> > > Ag
> > > AAACtw
> > >
> > > dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1w
> > > bH
> > > QAIURl
> > >
> > > bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4A
> > > DA
> > > IAAAI2
> > >
> > > cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4A
> > > JX
> > > EAfgAm
> > >
> > > c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJv
> > > eH
> > > l0ABpW
> > >
> > > bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAM
> > > AQ
> > > AAF09x
> > >
> > > AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52
> > > bS
> > > 5WbVdv
> > >
> > > cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9i
> > > cH
> > > BzcQB-
> > >
> > > AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2Jz
> > > Lm
> > > ltcGwu
> > >
> > > QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAM
> > > cn
> > > VuSW5D
> > >
> > > b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1h
> > > bm
> > > FnZWQu
> > >
> > > Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1
> > > bm
> > > 5hYmxl
> > >
> > > LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3Rh
> > > Y2
> > > subWFu
> > >
> > > YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0
> > > TW
> > > FuYWdl
> > >
> > > ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hl
> > > Lm
> > > Nsb3Vk
> > >
> > > c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-
> > > AE
> > > J0AA9j
> > >
> > > YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0
> > > aE
> > > NvbnRl
> > >
> > > eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdl
> > > ZC
> > > 5jb250
> > >
> > > ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEA
> > > fg
> > > AOcQB-
> > >
> > > ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5F
> > > eG
> > > VjdXRv
> > >
> > > cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZz
> > > cQ
> > > B-AAwC
> > >
> > > AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFz
> > > ay
> > > 5qYXZh
> > >
> > > cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQu
> > > VG
> > > hyZWFk
> > >
> > > UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJx
> > > AH
> > > 4AJXEA
> > >
> > > fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4
> > > ZW
> > > N1dG9y
> > >
> > > JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5n
> > > Ll
> > > RocmVh
> > >
> > > ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0
> > > aW
> > > 9ucyRF
> > > bXB0eUxpc3R6uBe0PKee3gIAAHhweA
> > >
> > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Publish async job-4915 complete on message bus
> > >
> > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Wake up jobs related to job-4915
> > >
> > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Update db status for job-4915
> > >
> > > 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Wake up jobs joined with job-4915 and disjoin all
> > > subjobs created from
> > > job- 4915
> > >
> > > 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Done executing com.cloud.vm.VmWorkStart for
> > > job-4915
> > >
> > > 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915)
> > > (logid:b59e4ee2) Remove
> > > job-4915 from job monitoring
> > >
> > > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for
> > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p
> > > 623 -U piotr -P
> > > Seneka22 chassis power status]: [Chassis Power is on
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error
> > > output command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status]:
> > > [Unable to Get Channel Cipher Suites
> > >
> > > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> > >
> > > Error response 0xc1 from Get PICMG Properities
> > >
> > > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> > >
> > > Invalid completion code received: Invalid command
> > >
> > > Discovered IPMB address 0x0
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,805 DEBUG
> > > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > > (pool-2-thread-10:null)
> > > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v
> > > -H
> > > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was
> > > successful and got the result [Chassis Power is on].
> > >
> > > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for
> > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p
> > > 623 -U piotr -P
> > > Seneka22 chassis power status]: [Chassis Power is on
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner]
> > > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error
> > > output command [/usr/bin/ipmitool -I lanplus -R 1 -v -H
> > > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status]:
> > > [Unable to Get Channel Cipher Suites
> > >
> > > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20
> > >
> > > Error response 0xc1 from Get PICMG Properities
> > >
> > > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20
> > >
> > > Invalid completion code received: Invalid command
> > >
> > > Discovered IPMB address 0x0
> > >
> > > ].
> > >
> > > 2021-11-17 14:47:27,806 DEBUG
> > > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver]
> > > (pool-2-thread-11:null)
> > > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v
> > > -H
> > > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was
> > > successful and got the result [Chassis Power is on].
> > >
> > > 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2)
> > > Unexpected exception while executing
> > > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin
> > >
> > > java.lang.ClassCastException: class com.cloud.agent.api.Answer
> > > cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriv
> > > eI
> > > so(Con
> > > figDriveNetworkElement.java:597)
> > >
> > > at
> > >
> > > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUs
> > > er
> > > data(C
> > > onfigDriveNetworkElement.java:225)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa
> > > re
> > > Elemen
> > > t(NetworkOrchestrator.java:1610)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa
> > > re
> > > Nic(Ne
> > > tworkOrchestrator.java:1936)
> > >
> > > at
> > >
> > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa
> > > re
> > > (Netwo
> > > rkOrchestrator.java:1870)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi
> > > ne
> > > Manage
> > > rImpl.java:1201)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi
> > > ne
> > > Manage
> > > rImpl.java:5802)
> > >
> > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati
> > > ve
> > > Method)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ
> > > eM
> > > ethodA
> > > ccessorImpl.java:62)
> > >
> > > at
> > >
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D
> > > el
> > > egatin
> > > gMethodAccessorImpl.java:43)
> > >
> > > at
> > > java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >
> > > at
> > >
> > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerP
> > > ro
> > > xy.jav
> > > a:107)
> > >
> > > at
> > >
> > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachin
> > > eM
> > > anager
> > > Impl.java:5967)
> > >
> > > at
> > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102
> > > )
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runI
> > > nC
> > > ontext
> > > (AsyncJobManagerImpl.java:620)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M
> > > an
> > > agedCo
> > > ntextRunnable.java:48)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c
> > > al
> > > l(Defa
> > > ultManagedContext.java:55)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal
> > > lW
> > > ithCon
> > > text(DefaultManagedContext.java:102)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run
> > > Wi
> > > thCont
> > > ext(DefaultManagedContext.java:52)
> > >
> > > at
> > >
> > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man
> > > ag
> > > edCont
> > > extRunnable.java:45)
> > >
> > > at
> > >
> > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(
> > > As
> > > yncJob
> > > ManagerImpl.java:568)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Execut
> > > or
> > > s.java
> > > :515)
> > >
> > > at
> > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPo
> > > ol
> > > Execut
> > > or.java:1128)
> > >
> > > at
> > >
> > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP
> > > oo
> > > lExecu
> > > tor.java:628)
> > >
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete
> > > async job-4912, jobStatus: FAILED, resultCode: 530, result:
> > >
> > > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> > > [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer
> > > cannot be cast to class
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer
> > > (com.cloud.agent.api.Answer and
> > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module
> > > of loader 'app')"}
> > >
> > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish
> > > async
> > > job-4912 complete on message bus
> > >
> > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > > jobs related to job-4912
> > >
> > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update
> > > db status for job-4912
> > >
> > > 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up
> > > jobs joined with job-4912 and disjoin all subjobs created
> > >
> > >
> >
> > --
> > Daan
> >
> >
> >
>
>
>
--
Daan