You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Granwille Strauss <gr...@namhost.com.INVALID> on 2022/12/20 06:08:39 UTC
Cannot create VM from Template Error
Hi Guys
I accidentality "deleted" a wrong template. Fortunately, the delete was
not successful and gave an error stating that it cannot be deleted
because there are active VMs associated with it. However, since I did
this, I can now no longer deploy new VMs using said template. Creating a
VM now fails since I did this, see attached screenshot.
And this is the management log entry:
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create a deployment for VM instance {id: "100", name: "i-2-100-VM",
> uuid: "e0611ff1-cab5-48d0-ac4e-09fe0f034b19",
> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job
> origin: 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null,
> lastUpdated: null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST
> 2022, removed: null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create a deployment for VM instance {id: "100", name: "i-2-100-VM",
> uuid: "e0611ff1-cab5-48d0-ac4e-09fe0f034b19",
> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
This seems to ONLY happen with this one template that I deleted, I can
deploy VMs successfully from other templates but this one that was
"deleted", even though it was NOT deleted. So it seems the template
broke somehow because now no volume is created for new VMs when using
this template. I am using 4.17.1.0. Is this a known issue?
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by Nux <nu...@li.nux.ro>.
Hi,
Can you open a GitHub issue for this please? Make sure to specify all
the relevant technical details, versions of stuff etc.
---
Nux
www.nux.ro [12]
On 2022-12-20 06:49, Granwille Strauss wrote:
> Hi
>
> I can confirm it is indeed deleted. So in short, the error that said
> something in the lines of You cannot delete template because there are
> active VMs associated with it, means nothing. It still deleted it
> anyway. I believe this needs to be looked at.
>
> On 12/20/22 08:45, jordan j wrote:
>
> Here is the issue, ACS cannot find the template volume to copy -
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId:
> 66988330791812, via: 36(athena03.namhost.com [1]), Ver: v1, Flags:
> 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux
> 8 cPanel
> Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4
> [2]","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00
> GB)
> 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4
> [2]","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB)
> 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been
> pending for 141 seconds
> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been
> pending for 201 seconds
> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been
> pending for 261 seconds
> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: ,
> MgmtId: 66988330791812, via: 36(athena03.namhost.com [1]), Ver: v1,
> Flags: 110, { CopyCmdAnswer } }
> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Unable to create
> Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> If other templates can be used from the same SR this means it is
> accessible but the template file is not there anymore.
> To look for it you can login on the secondary storage and look at a
> path similar to this <mount dir>/../templates.
> inside should be 1 folder for each template, it will include XML meta
> file and volume file. The volume, if exists should be named by the ID
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>
> if no such is present than indeed it is deleted.
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss
> <gr...@namhost.com> wrote:
>
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to confirm
> if a template exists on secondary storage?
>
> On 12/20/22 08:15, jordan j wrote:
>
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though
> it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary
> server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss
> <gr...@namhost.com.invalid> wrote:
>
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted
> because
> there are active VMs associated with it. However, since I did this, I
> can
> now no longer deploy new VMs using said template. Creating a VM now
> fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job
> origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null,
> lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022,
> removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template
> broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete
>
> <https://www.namhost.com> [3] Granwille Strauss // Senior Systems
> Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com [4]
>
> <https://www.facebook.com/namhost> [5] <https://twitter.com/namhost>
> [6]
> <https://www.instagram.com/namhostinternetservices/> [7]
> <https://www.linkedin.com/company/namhos> [8]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [9]
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [10]
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message.
> It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and
> security
> of this email cannot be guaranteed over the Internet. Therefore, the
> sender
> will not be held liable for any damage caused by the message. For our
> full
> privacy policy and disclaimers, please go to
> https://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [11]
--
Regards / Groete
[3]
Granwille Strauss // Senior Systems Admin
e: granwille@namhost.com
m: +264 81 323 1260 [13]
w: www.namhost.com [14]
[5] [6] [7] [8] [9]
[10]
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it
by mistake, please inform us by email reply and then delete the message.
It is forbidden to copy, forward, or in any way reveal the contents of
this message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
[11]
--
Regards / Groete
[3]
Granwille Strauss // Senior Systems Admin
e: granwille@namhost.com
m: +264 81 323 1260 [13]
w: www.namhost.com [14]
[5] [6] [7] [8] [9]
[10]
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it
by mistake, please inform us by email reply and then delete the message.
It is forbidden to copy, forward, or in any way reveal the contents of
this message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
[11]
Links:
------
[1] http://athena03.namhost.com
[2]
http://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4
[3] https://www.namhost.com
[4] http://www.namhost.com
[5] https://www.facebook.com/namhost
[6] https://twitter.com/namhost
[7] https://www.instagram.com/namhostinternetservices/
[8] https://www.linkedin.com/company/namhos
[9] https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA
[10]
https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner
[11]
https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818
[12] http://www.nux.ro
[13] tel:+264813231260
[14] https://www.namhost.com/
Re: Cannot create VM from Template Error
Posted by Nux <nu...@li.nux.ro>.
It's pretty bad, but there is a chance to recover data, an example of
how to get uncopulated here:
https://raddinox.com/restore-data-from-qcow2
HTH
---
Nux
www.nux.ro [11]
On 2022-12-22 13:40, Granwille Strauss wrote:
> Hi Guys
>
> If you delete a backing file, how screwed are you? Is it possible to
> recover the VM, because at this point the VM does not want to start.
>
>> "Cannot access backing file
>> '/mnt/3a808086-b3bd-3dda-b27d-18ab9f0059f4/d2c4feca-da5c-44b3-8ecc-40a01fd6c243'
>> of storage file
>> '/mnt/3a808086-b3bd-3dda-b27d-18ab9f0059f4/60edbb59-1a2a-47e0-a833-b21bc2a822a6'
>> (as uid:0, gid:0): No such file or
>> directory","wait":"0","bypassHostMaintenance":"false"}}] }
>
> On 12/22/22 12:52, Granwille Strauss wrote:
>
> Hi
>
> Managed to solve the last issue, seems I deleted an image file that
> caused this behaviour. Will log ticket on GitHub on initial issue.
>
> On 12/20/22 16:13, Wei ZHOU wrote:
>
> The logs shows
>
> com.cloud.utils.exception.CloudRuntimeException: Failed to copy
> /mnt/14205cea-e7ab-39c7-b8d1-0cea6cf65976/10666dea-0eae-3a43-a673-b07c587bf86b.qcow2
> to fa27384f-e4c8-4c34-8243-c32c7107b739
>
> Any error log on the host (id=36) ?
>
> -Wei
>
> On Tuesday, 20 December 2022, Granwille Strauss <gr...@namhost.com>
> wrote:
>
> Hi Guys
>
> I have not logged this on Github yet because I am still experiencing
> after
> affects caused by this bug. After we concluded that the image was
> indeed
> deleted. I "forced" deleted the template via CM and now tried to deploy
> a
> new template from a volume snapshot. This was successfully done thus
> far.
> Now when I want to boot a new VM using the new template I just created
> it
> keeps failing and I am seeing errors such as:
>
> Caused by: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for VM instance {id: "109", name:
> "i-2-109-VM", uuid: "43e31cda-4c8d-4c7d-8518-2a20d3b47d81",
> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
>
> 2022-12-20 15:06:45,646 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) DeploymentPlan has host_id specified, choosing this
> host
> and making no checks on this host: 36
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) The specified host is in avoid set
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) Cannot deploy to specified host, returning.
> 2022-12-20 15:06:45,655 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) VM instance {id: "109", name: "i-2-109-VM", uuid:
> "43e31cda-4c8d-4c7d-8518-2a20d3b47d81", type="User"} state transited
> from
> [Starting] to [Stopped] with event [OperationFailed]. VM's original
> host:
> null, new host: null, host before state transition: Host {"id": "36",
> "name": "hostname", "uuid": "35877892-f831-4dee-b6e2-19ed651f1299",
> "type"="Routing"}
>
> I am trying to deploy a new VM from a template created from a snapshot
> that exists in a different zone/host. I coped the template to my new
> zone
> and now trying to deploy said VM but seeing this failures. Attached is
> the
> full log during VM creation.
> Any ideas what can be causing this and potential options I can try to
> solve?
>
> On 12/20/22 12:58, Granwille Strauss wrote:
>
> Hi Wei
>
> I will log on Github. In the meantime, I processed "forced" delete for
> template so that I can get it removed from CM. Is it safe to remove
> images
> stored in primary, which are not related to VMs, since you mentioned
> templates are also stored in primary?
> On 12/20/22 12:48, Wei ZHOU wrote:
>
> This seems to be a bug. Please file a github issue.
>
> If the template is still used by active vms, you can find the template
> on
> primary storage.
>
> -Wei
>
> On Tue, 20 Dec 2022 at 07:50, Granwille
> Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid>
> wrote:
>
> Hi
>
> I can confirm it is indeed deleted. So in short, the error that said
> something in the lines of You cannot delete template because there are
> active VMs associated with it, means nothing. It still deleted it
> anyway. I
> believe this needs to be looked at.
> On 12/20/22 08:45, jordan j wrote:
>
> Here is the issue, ACS cannot find the template volume to copy -
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId:
> 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux
> 8 cPanel
> Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00
> GB)
> 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00
> GB)
> 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been
> pending for 141 seconds
> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been
> pending for 201 seconds
> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been
> pending for 261 seconds
> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: ,
> MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags:
> 110, { CopyCmdAnswer } }
> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Unable to create
> Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> If other templates can be used from the same SR this means it is
> accessible but the template file is not there anymore.
> To look for it you can login on the secondary storage and look at a
> path
> similar to this <mount dir>/../templates.
> inside should be 1 folder for each template, it will include XML meta
> file
> and volume file. The volume, if exists should be named by the ID
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>
> if no such is present than indeed it is deleted.
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss
> <gr...@namhost.com> <gr...@namhost.com>
> wrote:
>
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to confirm
> if a template exists on secondary storage?
> On 12/20/22 08:15, jordan j wrote:
>
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though
> it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary
> server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille
> Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid>
> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted
> because
> there are active VMs associated with it. However, since I did this, I
> can
> now no longer deploy new VMs using said template. Creating a VM now
> fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job
> origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null,
> lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022,
> removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template
> broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete<https://www.namhost.com> [1] <https://www.namhost.com>
> [1] <https://www.namhost.com> [1] <https://www.namhost.com> [1]
> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com [2]<https://www.facebook.com/namhost> [3]
> <https://www.facebook.com/namhost> [3]
> <https://www.facebook.com/namhost> [3]
> <https://www.facebook.com/namhost> [3] <https://twitter.com/namhost>
> [4] <https://twitter.com/namhost> [4] <https://twitter.com/namhost> [4]
> <https://twitter.com/namhost>
> [4]<https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.instagram.com/namhostinternetservices/>
> [5]<https://www.linkedin.com/company/namhos> [6]
> <https://www.linkedin.com/company/namhos> [6]
> <https://www.linkedin.com/company/namhos> [6]
> <https://www.linkedin.com/company/namhos>
> [6]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus
> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>
> [9], 7210, RSA
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message.
> It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and
> security
> of this email cannot be guaranteed over the Internet. Therefore, the
> sender
> will not be held liable for any damage caused by the message. For our
> full
> privacy policy and disclaimers, please go
> tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by
> AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
>
> --
> Regards / Groete
> <https://www.namhost.com> [1] <https://www.namhost.com> [1] Granwille
> Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com [2]
> <https://www.facebook.com/namhost> [3]
> <https://www.facebook.com/namhost> [3] <https://twitter.com/namhost>
> [4] <https://twitter.com/namhost>
> [4]<https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.instagram.com/namhostinternetservices/>
> [5]<https://www.linkedin.com/company/namhos> [6]
> <https://www.linkedin.com/company/namhos>
> [6]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus
> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>
> [9], 7210, RSA
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message.
> It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and
> security
> of this email cannot be guaranteed over the Internet. Therefore, the
> sender
> will not be held liable for any damage caused by the message. For our
> full
> privacy policy and disclaimers, please go
> tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by
> AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
>
> --
> Regards / Groete
> <https://www.namhost.com> [1] <https://www.namhost.com> [1] Granwille
> Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com [2]
> <https://www.facebook.com/namhost> [3]
> <https://www.facebook.com/namhost> [3] <https://twitter.com/namhost>
> [4] <https://twitter.com/namhost>
> [4]<https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.instagram.com/namhostinternetservices/>
> [5]<https://www.linkedin.com/company/namhos> [6]
> <https://www.linkedin.com/company/namhos>
> [6]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus
> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>
> [9], 7210, RSA
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message.
> It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and
> security
> of this email cannot be guaranteed over the Internet. Therefore, the
> sender
> will not be held liable for any damage caused by the message. For our
> full
> privacy policy and disclaimers, please go
> tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by
> AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
>
> --
> Regards / Groete
>
> <https://www.namhost.com> [1] Granwille Strauss // Senior Systems
> Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com [2]
>
> <https://www.facebook.com/namhost> [3] <https://twitter.com/namhost>
> [4]
> <https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.linkedin.com/company/namhos> [6]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus
> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>
> [9], 7210, RSA
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message.
> It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and
> security
> of this email cannot be guaranteed over the Internet. Therefore, the
> sender
> will not be held liable for any damage caused by the message. For our
> full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
>
> --
> Regards / Groete
>
> <https://www.namhost.com> [1] Granwille Strauss // Senior Systems
> Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com [2]
>
> <https://www.facebook.com/namhost> [3] <https://twitter.com/namhost>
> [4]
> <https://www.instagram.com/namhostinternetservices/> [5]
> <https://www.linkedin.com/company/namhos> [6]
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [7]
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> [8]
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus
> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>
> [9], 7210, RSA
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message.
> It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and
> security
> of this email cannot be guaranteed over the Internet. Therefore, the
> sender
> will not be held liable for any damage caused by the message. For our
> full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
> [10]
--
Regards / Groete
[1]
Granwille Strauss // Senior Systems Admin
e: granwille@namhost.com
m: +264 81 323 1260 [12]
w: www.namhost.com [13]
[3] [4] [5] [6] [7]
[8]
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it
by mistake, please inform us by email reply and then delete the message.
It is forbidden to copy, forward, or in any way reveal the contents of
this message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
[10]
--
Regards / Groete
[1]
Granwille Strauss // Senior Systems Admin
e: granwille@namhost.com
m: +264 81 323 1260 [12]
w: www.namhost.com [13]
[3] [4] [5] [6] [7]
[8]
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it
by mistake, please inform us by email reply and then delete the message.
It is forbidden to copy, forward, or in any way reveal the contents of
this message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
[10]
Links:
------
[1] https://www.namhost.com
[2] http://www.namhost.com
[3] https://www.facebook.com/namhost
[4] https://twitter.com/namhost
[5] https://www.instagram.com/namhostinternetservices/
[6] https://www.linkedin.com/company/namhos
[7] https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA
[8]
https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner
[9]
https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g
[10]
https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818
[11] http://www.nux.ro
[12] tel:+264813231260
[13] https://www.namhost.com/
Re: Cannot create VM from Template Error
Posted by Wei ZHOU <us...@gmail.com>.
If you can find the same image somewhere, you can copy it to the primary
storage with same filename
-Wei
On Thursday, 22 December 2022, Granwille Strauss <gr...@namhost.com>
wrote:
> Hi Guys
>
> If you delete a backing file, how screwed are you? Is it possible to
> recover the VM, because at this point the VM does not want to start.
>
> "Cannot access backing file '/mnt/3a808086-b3bd-3dda-b27d-
> 18ab9f0059f4/d2c4feca-da5c-44b3-8ecc-40a01fd6c243' of storage file
> '/mnt/3a808086-b3bd-3dda-b27d-18ab9f0059f4/60edbb59-1a2a-47e0-a833-b21bc2a822a6'
> (as uid:0, gid:0): No such file or directory","wait":"0","
> bypassHostMaintenance":"false"}}] }
>
>
> On 12/22/22 12:52, Granwille Strauss wrote:
>
> Hi
>
> Managed to solve the last issue, seems I deleted an image file that caused
> this behaviour. Will log ticket on GitHub on initial issue.
> On 12/20/22 16:13, Wei ZHOU wrote:
>
> The logs shows
>
> com.cloud.utils.exception.CloudRuntimeException: Failed to copy
> /mnt/14205cea-e7ab-39c7-b8d1-0cea6cf65976/10666dea-0eae-3a43-a673-b07c587bf86b.qcow2
> to fa27384f-e4c8-4c34-8243-c32c7107b739
>
> Any error log on the host (id=36) ?
>
> -Wei
>
>
> On Tuesday, 20 December 2022, Granwille Strauss <gr...@namhost.com> <gr...@namhost.com>
> wrote:
>
>
> Hi Guys
>
> I have not logged this on Github yet because I am still experiencing after
> affects caused by this bug. After we concluded that the image was indeed
> deleted. I "forced" deleted the template via CM and now tried to deploy a
> new template from a volume snapshot. This was successfully done thus far.
> Now when I want to boot a new VM using the new template I just created it
> keeps failing and I am seeing errors such as:
>
>
> Caused by: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for VM instance {id: "109", name:
> "i-2-109-VM", uuid: "43e31cda-4c8d-4c7d-8518-2a20d3b47d81",
> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
>
>
> 2022-12-20 15:06:45,646 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) DeploymentPlan has host_id specified, choosing this host
> and making no checks on this host: 36
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) The specified host is in avoid set
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) Cannot deploy to specified host, returning.
> 2022-12-20 15:06:45,655 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) VM instance {id: "109", name: "i-2-109-VM", uuid:
> "43e31cda-4c8d-4c7d-8518-2a20d3b47d81", type="User"} state transited from
> [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
> null, new host: null, host before state transition: Host {"id": "36",
> "name": "hostname", "uuid": "35877892-f831-4dee-b6e2-19ed651f1299",
> "type"="Routing"}
>
>
> I am trying to deploy a new VM from a template created from a snapshot
> that exists in a different zone/host. I coped the template to my new zone
> and now trying to deploy said VM but seeing this failures. Attached is the
> full log during VM creation.
> Any ideas what can be causing this and potential options I can try to
> solve?
>
> On 12/20/22 12:58, Granwille Strauss wrote:
>
> Hi Wei
>
> I will log on Github. In the meantime, I processed "forced" delete for
> template so that I can get it removed from CM. Is it safe to remove images
> stored in primary, which are not related to VMs, since you mentioned
> templates are also stored in primary?
> On 12/20/22 12:48, Wei ZHOU wrote:
>
> This seems to be a bug. Please file a github issue.
>
> If the template is still used by active vms, you can find the template on
> primary storage.
>
> -Wei
>
> On Tue, 20 Dec 2022 at 07:50, Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>
>
> Hi
>
> I can confirm it is indeed deleted. So in short, the error that said
> something in the lines of You cannot delete template because there are
> active VMs associated with it, means nothing. It still deleted it anyway. I
> believe this needs to be looked at.
> On 12/20/22 08:45, jordan j wrote:
>
> Here is the issue, ACS cannot find the template volume to copy -
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
>
> If other templates can be used from the same SR this means it is
> accessible but the template file is not there anymore.
> To look for it you can login on the secondary storage and look at a path
> similar to this <mount dir>/../templates.
> inside should be 1 folder for each template, it will include XML meta file
> and volume file. The volume, if exists should be named by the ID
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>
> if no such is present than indeed it is deleted.
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss <gr...@namhost.com> <gr...@namhost.com> <gr...@namhost.com> <gr...@namhost.com>
> wrote:
>
>
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to confirm
> if a template exists on secondary storage?
> On 12/20/22 08:15, jordan j wrote:
>
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>
>
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted because
> there are active VMs associated with it. However, since I did this, I can
> now no longer deploy new VMs using said template. Creating a VM now fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete<https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com<https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA><https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete<https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com<https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete<https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com<https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g> <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
Re: Cannot create VM from Template Error
Posted by Granwille Strauss <gr...@namhost.com.INVALID>.
Hi Guys
If you delete a backing file, how screwed are you? Is it possible to
recover the VM, because at this point the VM does not want to start.
> "Cannot access backing file
> '/mnt/3a808086-b3bd-3dda-b27d-18ab9f0059f4/d2c4feca-da5c-44b3-8ecc-40a01fd6c243'
> of storage file
> '/mnt/3a808086-b3bd-3dda-b27d-18ab9f0059f4/60edbb59-1a2a-47e0-a833-b21bc2a822a6'
> (as uid:0, gid:0): No such file or
> directory","wait":"0","bypassHostMaintenance":"false"}}] }
On 12/22/22 12:52, Granwille Strauss wrote:
>
> Hi
>
> Managed to solve the last issue, seems I deleted an image file that
> caused this behaviour. Will log ticket on GitHub on initial issue.
>
> On 12/20/22 16:13, Wei ZHOU wrote:
>> The logs shows
>>
>> com.cloud.utils.exception.CloudRuntimeException: Failed to copy
>> /mnt/14205cea-e7ab-39c7-b8d1-0cea6cf65976/10666dea-0eae-3a43-a673-b07c587bf86b.qcow2
>> to fa27384f-e4c8-4c34-8243-c32c7107b739
>>
>> Any error log on the host (id=36) ?
>>
>> -Wei
>>
>>
>> On Tuesday, 20 December 2022, Granwille Strauss<gr...@namhost.com>
>> wrote:
>>
>>> Hi Guys
>>>
>>> I have not logged this on Github yet because I am still experiencing after
>>> affects caused by this bug. After we concluded that the image was indeed
>>> deleted. I "forced" deleted the template via CM and now tried to deploy a
>>> new template from a volume snapshot. This was successfully done thus far.
>>> Now when I want to boot a new VM using the new template I just created it
>>> keeps failing and I am seeing errors such as:
>>>
>>>
>>> Caused by: com.cloud.exception.InsufficientServerCapacityException:
>>> Unable to create a deployment for VM instance {id: "109", name:
>>> "i-2-109-VM", uuid: "43e31cda-4c8d-4c7d-8518-2a20d3b47d81",
>>> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
>>>
>>>
>>> 2022-12-20 15:06:45,646 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>>> (logid:769d88da) DeploymentPlan has host_id specified, choosing this host
>>> and making no checks on this host: 36
>>> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>>> (logid:769d88da) The specified host is in avoid set
>>> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>>> (logid:769d88da) Cannot deploy to specified host, returning.
>>> 2022-12-20 15:06:45,655 DEBUG [c.c.c.CapacityManagerImpl]
>>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>>> (logid:769d88da) VM instance {id: "109", name: "i-2-109-VM", uuid:
>>> "43e31cda-4c8d-4c7d-8518-2a20d3b47d81", type="User"} state transited from
>>> [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
>>> null, new host: null, host before state transition: Host {"id": "36",
>>> "name": "hostname", "uuid": "35877892-f831-4dee-b6e2-19ed651f1299",
>>> "type"="Routing"}
>>>
>>>
>>> I am trying to deploy a new VM from a template created from a snapshot
>>> that exists in a different zone/host. I coped the template to my new zone
>>> and now trying to deploy said VM but seeing this failures. Attached is the
>>> full log during VM creation.
>>> Any ideas what can be causing this and potential options I can try to
>>> solve?
>>>
>>> On 12/20/22 12:58, Granwille Strauss wrote:
>>>
>>> Hi Wei
>>>
>>> I will log on Github. In the meantime, I processed "forced" delete for
>>> template so that I can get it removed from CM. Is it safe to remove images
>>> stored in primary, which are not related to VMs, since you mentioned
>>> templates are also stored in primary?
>>> On 12/20/22 12:48, Wei ZHOU wrote:
>>>
>>> This seems to be a bug. Please file a github issue.
>>>
>>> If the template is still used by active vms, you can find the template on
>>> primary storage.
>>>
>>> -Wei
>>>
>>> On Tue, 20 Dec 2022 at 07:50, Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>>
>>>
>>> Hi
>>>
>>> I can confirm it is indeed deleted. So in short, the error that said
>>> something in the lines of You cannot delete template because there are
>>> active VMs associated with it, means nothing. It still deleted it anyway. I
>>> believe this needs to be looked at.
>>> On 12/20/22 08:45, jordan j wrote:
>>>
>>> Here is the issue, ACS cannot find the template volume to copy -
>>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>>
>>> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
>>> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
>>> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
>>> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
>>> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
>>> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>>
>>>
>>> If other templates can be used from the same SR this means it is
>>> accessible but the template file is not there anymore.
>>> To look for it you can login on the secondary storage and look at a path
>>> similar to this <mount dir>/../templates.
>>> inside should be 1 folder for each template, it will include XML meta file
>>> and volume file. The volume, if exists should be named by the ID
>>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>>>
>>> if no such is present than indeed it is deleted.
>>>
>>> Regards,
>>> Jordan
>>>
>>> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss<gr...@namhost.com> <gr...@namhost.com>
>>> wrote:
>>>
>>>
>>> Hi Jordan
>>>
>>> Please see log attached. Can you confirm what's the best way to confirm
>>> if a template exists on secondary storage?
>>> On 12/20/22 08:15, jordan j wrote:
>>>
>>> Hey Granwille Strauss,
>>>
>>> Can you upload a full log somewhere?
>>> This error usually means there is another underlying issue even though it
>>> says Network/interface scope.
>>> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>>
>>> Can you also verify the template file is intact on the secondary server?
>>>
>>> Regards,
>>> Jordan
>>>
>>> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>>
>>>
>>> Hi Guys
>>>
>>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>>> not successful and gave an error stating that it cannot be deleted because
>>> there are active VMs associated with it. However, since I did this, I can
>>> now no longer deploy new VMs using said template. Creating a VM now fails
>>> since I did this, see attached screenshot.
>>>
>>> And this is the management log entry:
>>>
>>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>>> (logid:4c740aa8) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>>> 2783
>>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>>> cmdInfo:
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>>> null}, job origin:2783
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>>
>>> This seems to ONLY happen with this one template that I deleted, I can
>>> deploy VMs successfully from other templates but this one that was
>>> "deleted", even though it was NOT deleted. So it seems the template broke
>>> somehow because now no volume is created for new VMs when using this
>>> template. I am using 4.17.1.0. Is this a known issue?
>>> --
>>> Regards / Groete<https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com<https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
>>> --
>>> Regards / Groete
>>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
>>> --
>>> Regards / Groete
>>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
>>> --
>>> Regards / Groete
>>>
>>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>>
>>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>>> <https://www.instagram.com/namhostinternetservices/>
>>> <https://www.linkedin.com/company/namhos>
>>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>>
>>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/
>>> privacy-policy
>>>
>>> [image: Powered by AdSigner]
>>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
>>> --
>>> Regards / Groete
>>>
>>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>>
>>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>>> <https://www.instagram.com/namhostinternetservices/>
>>> <https://www.linkedin.com/company/namhos>
>>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>>
>>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/
>>> privacy-policy
>>>
>>> [image: Powered by AdSigner]
>>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <tel:+264813231260>
> *w:* www.namhost.com <https://www.namhost.com/>
>
> <https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
> The content of this message is confidential. If you have received it
> by mistake, please inform us by email reply and then delete the
> message. It is forbidden to copy, forward, or in any way reveal the
> contents of this message to anyone without our explicit consent. The
> integrity and security of this email cannot be guaranteed over the
> Internet. Therefore, the sender will not be held liable for any damage
> caused by the message. For our full privacy policy and disclaimers,
> please go to https://www.namhost.com/privacy-policy
>
> Powered by AdSigner
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by Granwille Strauss <gr...@namhost.com.INVALID>.
Hi
Managed to solve the last issue, seems I deleted an image file that
caused this behaviour. Will log ticket on GitHub on initial issue.
On 12/20/22 16:13, Wei ZHOU wrote:
> The logs shows
>
> com.cloud.utils.exception.CloudRuntimeException: Failed to copy
> /mnt/14205cea-e7ab-39c7-b8d1-0cea6cf65976/10666dea-0eae-3a43-a673-b07c587bf86b.qcow2
> to fa27384f-e4c8-4c34-8243-c32c7107b739
>
> Any error log on the host (id=36) ?
>
> -Wei
>
>
> On Tuesday, 20 December 2022, Granwille Strauss<gr...@namhost.com>
> wrote:
>
>> Hi Guys
>>
>> I have not logged this on Github yet because I am still experiencing after
>> affects caused by this bug. After we concluded that the image was indeed
>> deleted. I "forced" deleted the template via CM and now tried to deploy a
>> new template from a volume snapshot. This was successfully done thus far.
>> Now when I want to boot a new VM using the new template I just created it
>> keeps failing and I am seeing errors such as:
>>
>>
>> Caused by: com.cloud.exception.InsufficientServerCapacityException:
>> Unable to create a deployment for VM instance {id: "109", name:
>> "i-2-109-VM", uuid: "43e31cda-4c8d-4c7d-8518-2a20d3b47d81",
>> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
>>
>>
>> 2022-12-20 15:06:45,646 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>> (logid:769d88da) DeploymentPlan has host_id specified, choosing this host
>> and making no checks on this host: 36
>> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>> (logid:769d88da) The specified host is in avoid set
>> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>> (logid:769d88da) Cannot deploy to specified host, returning.
>> 2022-12-20 15:06:45,655 DEBUG [c.c.c.CapacityManagerImpl]
>> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
>> (logid:769d88da) VM instance {id: "109", name: "i-2-109-VM", uuid:
>> "43e31cda-4c8d-4c7d-8518-2a20d3b47d81", type="User"} state transited from
>> [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
>> null, new host: null, host before state transition: Host {"id": "36",
>> "name": "hostname", "uuid": "35877892-f831-4dee-b6e2-19ed651f1299",
>> "type"="Routing"}
>>
>>
>> I am trying to deploy a new VM from a template created from a snapshot
>> that exists in a different zone/host. I coped the template to my new zone
>> and now trying to deploy said VM but seeing this failures. Attached is the
>> full log during VM creation.
>> Any ideas what can be causing this and potential options I can try to
>> solve?
>>
>> On 12/20/22 12:58, Granwille Strauss wrote:
>>
>> Hi Wei
>>
>> I will log on Github. In the meantime, I processed "forced" delete for
>> template so that I can get it removed from CM. Is it safe to remove images
>> stored in primary, which are not related to VMs, since you mentioned
>> templates are also stored in primary?
>> On 12/20/22 12:48, Wei ZHOU wrote:
>>
>> This seems to be a bug. Please file a github issue.
>>
>> If the template is still used by active vms, you can find the template on
>> primary storage.
>>
>> -Wei
>>
>> On Tue, 20 Dec 2022 at 07:50, Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>
>>
>> Hi
>>
>> I can confirm it is indeed deleted. So in short, the error that said
>> something in the lines of You cannot delete template because there are
>> active VMs associated with it, means nothing. It still deleted it anyway. I
>> believe this needs to be looked at.
>> On 12/20/22 08:45, jordan j wrote:
>>
>> Here is the issue, ACS cannot find the template volume to copy -
>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>
>> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
>> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
>> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
>> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
>> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
>> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>
>>
>> If other templates can be used from the same SR this means it is
>> accessible but the template file is not there anymore.
>> To look for it you can login on the secondary storage and look at a path
>> similar to this <mount dir>/../templates.
>> inside should be 1 folder for each template, it will include XML meta file
>> and volume file. The volume, if exists should be named by the ID
>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>>
>> if no such is present than indeed it is deleted.
>>
>> Regards,
>> Jordan
>>
>> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss<gr...@namhost.com> <gr...@namhost.com>
>> wrote:
>>
>>
>> Hi Jordan
>>
>> Please see log attached. Can you confirm what's the best way to confirm
>> if a template exists on secondary storage?
>> On 12/20/22 08:15, jordan j wrote:
>>
>> Hey Granwille Strauss,
>>
>> Can you upload a full log somewhere?
>> This error usually means there is another underlying issue even though it
>> says Network/interface scope.
>> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>>
>> Can you also verify the template file is intact on the secondary server?
>>
>> Regards,
>> Jordan
>>
>> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>
>>
>> Hi Guys
>>
>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>> not successful and gave an error stating that it cannot be deleted because
>> there are active VMs associated with it. However, since I did this, I can
>> now no longer deploy new VMs using said template. Creating a VM now fails
>> since I did this, see attached screenshot.
>>
>> And this is the management log entry:
>>
>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>> (logid:4c740aa8) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>> 2783
>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>> cmdInfo:
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>> null}, job origin:2783
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>>
>> This seems to ONLY happen with this one template that I deleted, I can
>> deploy VMs successfully from other templates but this one that was
>> "deleted", even though it was NOT deleted. So it seems the template broke
>> somehow because now no volume is created for new VMs when using this
>> template. I am using 4.17.1.0. Is this a known issue?
>> --
>> Regards / Groete<https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com<https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
>> --
>> Regards / Groete
>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com
>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
>> --
>> Regards / Groete
>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com
>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
>> --
>> Regards / Groete
>>
>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com
>>
>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>> <https://www.instagram.com/namhostinternetservices/>
>> <https://www.linkedin.com/company/namhos>
>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>>
>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go tohttps://www.namhost.com/
>> privacy-policy
>>
>> [image: Powered by AdSigner]
>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
>> --
>> Regards / Groete
>>
>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com
>>
>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>> <https://www.instagram.com/namhostinternetservices/>
>> <https://www.linkedin.com/company/namhos>
>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>>
>> 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go tohttps://www.namhost.com/
>> privacy-policy
>>
>> [image: Powered by AdSigner]
>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by Wei ZHOU <us...@gmail.com>.
The logs shows
com.cloud.utils.exception.CloudRuntimeException: Failed to copy
/mnt/14205cea-e7ab-39c7-b8d1-0cea6cf65976/10666dea-0eae-3a43-a673-b07c587bf86b.qcow2
to fa27384f-e4c8-4c34-8243-c32c7107b739
Any error log on the host (id=36) ?
-Wei
On Tuesday, 20 December 2022, Granwille Strauss <gr...@namhost.com>
wrote:
> Hi Guys
>
> I have not logged this on Github yet because I am still experiencing after
> affects caused by this bug. After we concluded that the image was indeed
> deleted. I "forced" deleted the template via CM and now tried to deploy a
> new template from a volume snapshot. This was successfully done thus far.
> Now when I want to boot a new VM using the new template I just created it
> keeps failing and I am seeing errors such as:
>
>
> Caused by: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for VM instance {id: "109", name:
> "i-2-109-VM", uuid: "43e31cda-4c8d-4c7d-8518-2a20d3b47d81",
> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
>
>
> 2022-12-20 15:06:45,646 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) DeploymentPlan has host_id specified, choosing this host
> and making no checks on this host: 36
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) The specified host is in avoid set
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) Cannot deploy to specified host, returning.
> 2022-12-20 15:06:45,655 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) VM instance {id: "109", name: "i-2-109-VM", uuid:
> "43e31cda-4c8d-4c7d-8518-2a20d3b47d81", type="User"} state transited from
> [Starting] to [Stopped] with event [OperationFailed]. VM's original host:
> null, new host: null, host before state transition: Host {"id": "36",
> "name": "hostname", "uuid": "35877892-f831-4dee-b6e2-19ed651f1299",
> "type"="Routing"}
>
>
> I am trying to deploy a new VM from a template created from a snapshot
> that exists in a different zone/host. I coped the template to my new zone
> and now trying to deploy said VM but seeing this failures. Attached is the
> full log during VM creation.
> Any ideas what can be causing this and potential options I can try to
> solve?
>
> On 12/20/22 12:58, Granwille Strauss wrote:
>
> Hi Wei
>
> I will log on Github. In the meantime, I processed "forced" delete for
> template so that I can get it removed from CM. Is it safe to remove images
> stored in primary, which are not related to VMs, since you mentioned
> templates are also stored in primary?
> On 12/20/22 12:48, Wei ZHOU wrote:
>
> This seems to be a bug. Please file a github issue.
>
> If the template is still used by active vms, you can find the template on
> primary storage.
>
> -Wei
>
> On Tue, 20 Dec 2022 at 07:50, Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>
>
> Hi
>
> I can confirm it is indeed deleted. So in short, the error that said
> something in the lines of You cannot delete template because there are
> active VMs associated with it, means nothing. It still deleted it anyway. I
> believe this needs to be looked at.
> On 12/20/22 08:45, jordan j wrote:
>
> Here is the issue, ACS cannot find the template volume to copy -
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
>
> If other templates can be used from the same SR this means it is
> accessible but the template file is not there anymore.
> To look for it you can login on the secondary storage and look at a path
> similar to this <mount dir>/../templates.
> inside should be 1 folder for each template, it will include XML meta file
> and volume file. The volume, if exists should be named by the ID
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>
> if no such is present than indeed it is deleted.
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss <gr...@namhost.com> <gr...@namhost.com>
> wrote:
>
>
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to confirm
> if a template exists on secondary storage?
> On 12/20/22 08:15, jordan j wrote:
>
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>
>
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted because
> there are active VMs associated with it. However, since I did this, I can
> now no longer deploy new VMs using said template. Creating a VM now fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete<https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com<https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus <https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to https://www.namhost.com/
> privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
Re: Cannot create VM from Template Error
Posted by Granwille Strauss <gr...@namhost.com.INVALID>.
Hi Guys
I have not logged this on Github yet because I am still experiencing
after affects caused by this bug. After we concluded that the image was
indeed deleted. I "forced" deleted the template via CM and now tried to
deploy a new template from a volume snapshot. This was successfully done
thus far. Now when I want to boot a new VM using the new template I
just created it keeps failing and I am seeing errors such as:
> Caused by: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for VM instance {id: "109", name:
> "i-2-109-VM", uuid: "43e31cda-4c8d-4c7d-8518-2a20d3b47d81",
> type="User"}Scope=interface com.cloud.dc.DataCenter; id=16
> 2022-12-20 15:06:45,646 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) DeploymentPlan has host_id specified, choosing this
> host and making no checks on this host: 36
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) The specified host is in avoid set
> 2022-12-20 15:06:45,647 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) Cannot deploy to specified host, returning.
> 2022-12-20 15:06:45,655 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-24:ctx-15a7865f job-2860/job-2861 ctx-a0108f4c)
> (logid:769d88da) VM instance {id: "109", name: "i-2-109-VM", uuid:
> "43e31cda-4c8d-4c7d-8518-2a20d3b47d81", type="User"} state transited
> from [Starting] to [Stopped] with event [OperationFailed]. VM's
> original host: null, new host: null, host before state transition:
> Host {"id": "36", "name": "hostname", "uuid":
> "35877892-f831-4dee-b6e2-19ed651f1299", "type"="Routing"}
I am trying to deploy a new VM from a template created from a snapshot
that exists in a different zone/host. I coped the template to my new
zone and now trying to deploy said VM but seeing this failures. Attached
is the full log during VM creation.
Any ideas what can be causing this and potential options I can try to
solve?
On 12/20/22 12:58, Granwille Strauss wrote:
>
> Hi Wei
>
> I will log on Github. In the meantime, I processed "forced" delete for
> template so that I can get it removed from CM. Is it safe to remove
> images stored in primary, which are not related to VMs, since you
> mentioned templates are also stored in primary?
>
> On 12/20/22 12:48, Wei ZHOU wrote:
>> This seems to be a bug. Please file a github issue.
>>
>> If the template is still used by active vms, you can find the template on
>> primary storage.
>>
>> -Wei
>>
>> On Tue, 20 Dec 2022 at 07:50, Granwille Strauss
>> <gr...@namhost.com.invalid> wrote:
>>
>>> Hi
>>>
>>> I can confirm it is indeed deleted. So in short, the error that said
>>> something in the lines of You cannot delete template because there are
>>> active VMs associated with it, means nothing. It still deleted it anyway. I
>>> believe this needs to be looked at.
>>> On 12/20/22 08:45, jordan j wrote:
>>>
>>> Here is the issue, ACS cannot find the template volume to copy -
>>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>>
>>> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
>>> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
>>> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
>>> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
>>> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
>>> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>>
>>>
>>> If other templates can be used from the same SR this means it is
>>> accessible but the template file is not there anymore.
>>> To look for it you can login on the secondary storage and look at a path
>>> similar to this <mount dir>/../templates.
>>> inside should be 1 folder for each template, it will include XML meta file
>>> and volume file. The volume, if exists should be named by the ID
>>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>>>
>>> if no such is present than indeed it is deleted.
>>>
>>> Regards,
>>> Jordan
>>>
>>> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss<gr...@namhost.com>
>>> wrote:
>>>
>>>> Hi Jordan
>>>>
>>>> Please see log attached. Can you confirm what's the best way to confirm
>>>> if a template exists on secondary storage?
>>>> On 12/20/22 08:15, jordan j wrote:
>>>>
>>>> Hey Granwille Strauss,
>>>>
>>>> Can you upload a full log somewhere?
>>>> This error usually means there is another underlying issue even though it
>>>> says Network/interface scope.
>>>> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>>> com.cloud.dc.DataCenter; id=16
>>>>
>>>> Can you also verify the template file is intact on the secondary server?
>>>>
>>>> Regards,
>>>> Jordan
>>>>
>>>> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>>>
>>>>
>>>> Hi Guys
>>>>
>>>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>>>> not successful and gave an error stating that it cannot be deleted because
>>>> there are active VMs associated with it. However, since I did this, I can
>>>> now no longer deploy new VMs using said template. Creating a VM now fails
>>>> since I did this, see attached screenshot.
>>>>
>>>> And this is the management log entry:
>>>>
>>>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>>>> (logid:4c740aa8) Rethrow exception
>>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>>>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>>> com.cloud.dc.DataCenter; id=16
>>>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>>>> 2783
>>>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>>>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>>>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>>>> cmdInfo:
>>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>>>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>>>> null}, job origin:2783
>>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>>>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>>> com.cloud.dc.DataCenter; id=16
>>>>
>>>> This seems to ONLY happen with this one template that I deleted, I can
>>>> deploy VMs successfully from other templates but this one that was
>>>> "deleted", even though it was NOT deleted. So it seems the template broke
>>>> somehow because now no volume is created for new VMs when using this
>>>> template. I am using 4.17.1.0. Is this a known issue?
>>>> --
>>>> Regards / Groete
>>>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>>
>>>> *e:*granwille@namhost.com
>>>> *m:* +264 81 323 1260 <+264813231260>
>>>> *w:*www.namhost.com
>>>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>>
>>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>>
>>>> Namhost Internet Services (Pty) Ltd,
>>>>
>>>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>>>
>>>>
>>>>
>>>> The content of this message is confidential. If you have received it by
>>>> mistake, please inform us by email reply and then delete the message. It is
>>>> forbidden to copy, forward, or in any way reveal the contents of this
>>>> message to anyone without our explicit consent. The integrity and security
>>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>>> will not be held liable for any damage caused by the message. For our full
>>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>>>
>>>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>>
>>>> --
>>>> Regards / Groete
>>>>
>>>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>>
>>>> *e:*granwille@namhost.com
>>>> *m:* +264 81 323 1260 <+264813231260>
>>>> *w:*www.namhost.com
>>>>
>>>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>>>> <https://www.instagram.com/namhostinternetservices/>
>>>> <https://www.linkedin.com/company/namhos>
>>>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>>
>>>>
>>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>>
>>>> Namhost Internet Services (Pty) Ltd,
>>>>
>>>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>>>
>>>>
>>>>
>>>> The content of this message is confidential. If you have received it by
>>>> mistake, please inform us by email reply and then delete the message. It is
>>>> forbidden to copy, forward, or in any way reveal the contents of this
>>>> message to anyone without our explicit consent. The integrity and security
>>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>>> will not be held liable for any damage caused by the message. For our full
>>>> privacy policy and disclaimers, please go to
>>>> https://www.namhost.com/privacy-policy
>>>>
>>>> [image: Powered by AdSigner]
>>>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>>
>>> --
>>> Regards / Groete
>>>
>>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>>
>>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>>> <https://www.instagram.com/namhostinternetservices/>
>>> <https://www.linkedin.com/company/namhos>
>>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>>
>>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go to
>>> https://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]
>>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <tel:+264813231260>
> *w:* www.namhost.com <https://www.namhost.com/>
>
> <https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
> The content of this message is confidential. If you have received it
> by mistake, please inform us by email reply and then delete the
> message. It is forbidden to copy, forward, or in any way reveal the
> contents of this message to anyone without our explicit consent. The
> integrity and security of this email cannot be guaranteed over the
> Internet. Therefore, the sender will not be held liable for any damage
> caused by the message. For our full privacy policy and disclaimers,
> please go to https://www.namhost.com/privacy-policy
>
> Powered by AdSigner
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by Granwille Strauss <gr...@namhost.com.INVALID>.
Hi Wei
I will log on Github. In the meantime, I processed "forced" delete for
template so that I can get it removed from CM. Is it safe to remove
images stored in primary, which are not related to VMs, since you
mentioned templates are also stored in primary?
On 12/20/22 12:48, Wei ZHOU wrote:
> This seems to be a bug. Please file a github issue.
>
> If the template is still used by active vms, you can find the template on
> primary storage.
>
> -Wei
>
> On Tue, 20 Dec 2022 at 07:50, Granwille Strauss
> <gr...@namhost.com.invalid> wrote:
>
>> Hi
>>
>> I can confirm it is indeed deleted. So in short, the error that said
>> something in the lines of You cannot delete template because there are
>> active VMs associated with it, means nothing. It still deleted it anyway. I
>> believe this needs to be looked at.
>> On 12/20/22 08:45, jordan j wrote:
>>
>> Here is the issue, ACS cannot find the template volume to copy -
>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>
>> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
>> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
>> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
>> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
>> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
>> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>>
>>
>> If other templates can be used from the same SR this means it is
>> accessible but the template file is not there anymore.
>> To look for it you can login on the secondary storage and look at a path
>> similar to this <mount dir>/../templates.
>> inside should be 1 folder for each template, it will include XML meta file
>> and volume file. The volume, if exists should be named by the ID
>> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>>
>> if no such is present than indeed it is deleted.
>>
>> Regards,
>> Jordan
>>
>> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss<gr...@namhost.com>
>> wrote:
>>
>>> Hi Jordan
>>>
>>> Please see log attached. Can you confirm what's the best way to confirm
>>> if a template exists on secondary storage?
>>> On 12/20/22 08:15, jordan j wrote:
>>>
>>> Hey Granwille Strauss,
>>>
>>> Can you upload a full log somewhere?
>>> This error usually means there is another underlying issue even though it
>>> says Network/interface scope.
>>> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>>
>>> Can you also verify the template file is intact on the secondary server?
>>>
>>> Regards,
>>> Jordan
>>>
>>> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>>
>>>
>>> Hi Guys
>>>
>>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>>> not successful and gave an error stating that it cannot be deleted because
>>> there are active VMs associated with it. However, since I did this, I can
>>> now no longer deploy new VMs using said template. Creating a VM now fails
>>> since I did this, see attached screenshot.
>>>
>>> And this is the management log entry:
>>>
>>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>>> (logid:4c740aa8) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>>> 2783
>>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>>> cmdInfo:
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>>> null}, job origin:2783
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>>
>>> This seems to ONLY happen with this one template that I deleted, I can
>>> deploy VMs successfully from other templates but this one that was
>>> "deleted", even though it was NOT deleted. So it seems the template broke
>>> somehow because now no volume is created for new VMs when using this
>>> template. I am using 4.17.1.0. Is this a known issue?
>>> --
>>> Regards / Groete
>>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>>
>>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
>>> --
>>> Regards / Groete
>>>
>>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com
>>>
>>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>>> <https://www.instagram.com/namhostinternetservices/>
>>> <https://www.linkedin.com/company/namhos>
>>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>>
>>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go to
>>> https://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]
>>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
>> --
>> Regards / Groete
>>
>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com
>>
>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>> <https://www.instagram.com/namhostinternetservices/>
>> <https://www.linkedin.com/company/namhos>
>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>>
>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go to
>> https://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]
>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by Wei ZHOU <us...@gmail.com>.
This seems to be a bug. Please file a github issue.
If the template is still used by active vms, you can find the template on
primary storage.
-Wei
On Tue, 20 Dec 2022 at 07:50, Granwille Strauss
<gr...@namhost.com.invalid> wrote:
> Hi
>
> I can confirm it is indeed deleted. So in short, the error that said
> something in the lines of You cannot delete template because there are
> active VMs associated with it, means nothing. It still deleted it anyway. I
> believe this needs to be looked at.
> On 12/20/22 08:45, jordan j wrote:
>
> Here is the issue, ACS cannot find the template volume to copy -
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 110, { CopyCmdAnswer } }
> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
>
> If other templates can be used from the same SR this means it is
> accessible but the template file is not there anymore.
> To look for it you can login on the secondary storage and look at a path
> similar to this <mount dir>/../templates.
> inside should be 1 folder for each template, it will include XML meta file
> and volume file. The volume, if exists should be named by the ID
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>
> if no such is present than indeed it is deleted.
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss <gr...@namhost.com>
> wrote:
>
>> Hi Jordan
>>
>> Please see log attached. Can you confirm what's the best way to confirm
>> if a template exists on secondary storage?
>> On 12/20/22 08:15, jordan j wrote:
>>
>> Hey Granwille Strauss,
>>
>> Can you upload a full log somewhere?
>> This error usually means there is another underlying issue even though it
>> says Network/interface scope.
>> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>>
>> Can you also verify the template file is intact on the secondary server?
>>
>> Regards,
>> Jordan
>>
>> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>>
>>
>> Hi Guys
>>
>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>> not successful and gave an error stating that it cannot be deleted because
>> there are active VMs associated with it. However, since I did this, I can
>> now no longer deploy new VMs using said template. Creating a VM now fails
>> since I did this, see attached screenshot.
>>
>> And this is the management log entry:
>>
>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>> (logid:4c740aa8) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>> 2783
>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>> cmdInfo:
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>> null}, job origin:2783
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>>
>> This seems to ONLY happen with this one template that I deleted, I can
>> deploy VMs successfully from other templates but this one that was
>> "deleted", even though it was NOT deleted. So it seems the template broke
>> somehow because now no volume is created for new VMs when using this
>> template. I am using 4.17.1.0. Is this a known issue?
>> --
>> Regards / Groete
>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:* granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:* www.namhost.com
>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>>
>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
>> --
>> Regards / Groete
>>
>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:* granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:* www.namhost.com
>>
>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>> <https://www.instagram.com/namhostinternetservices/>
>> <https://www.linkedin.com/company/namhos>
>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>>
>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go to
>> https://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]
>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to
> https://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
Re: Cannot create VM from Template Error
Posted by Granwille Strauss <gr...@namhost.com.INVALID>.
Hi
I can confirm it is indeed deleted. So in short, the error that said
something in the lines of You cannot delete template because there are
active VMs associated with it, means nothing. It still deleted it
anyway. I believe this needs to be looked at.
On 12/20/22 08:45, jordan j wrote:
> Here is the issue, ACS cannot find the template volume to copy -
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243
> 2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId: 66988330791812, via: 36(athena03.namhost.com <http://athena03.namhost.com>), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux 8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4 <http://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4>","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00 GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4 <http://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4>","isManaged":"false"}},"name":"ROOT-100","size":"(60,00 GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}] }
> 2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been pending for 141 seconds
> 2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been pending for 201 seconds
> 2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been pending for 261 seconds
> 2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: , MgmtId: 66988330791812, via: 36(athena03.namhost.com <http://athena03.namhost.com>), Ver: v1, Flags: 110, { CopyCmdAnswer } }
> 2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481) (logid:4c740aa8) Unable to create Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
>
> If other templates can be used from the same SR this means it is
> accessible but the template file is not there anymore.
> To look for it you can login on the secondary storage and look at a
> path similar to this <mount dir>/../templates.
> inside should be 1 folder for each template, it will include XML meta
> file and volume file. The volume, if exists should be named by the ID
> d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
>
> if no such is present than indeed it is deleted.
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss
> <gr...@namhost.com> wrote:
>
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to
> confirm if a template exists on secondary storage?
>
> On 12/20/22 08:15, jordan j wrote:
>> Hey Granwille Strauss,
>>
>> Can you upload a full log somewhere?
>> This error usually means there is another underlying issue even though it
>> says Network/interface scope.
>> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>>
>> Can you also verify the template file is intact on the secondary server?
>>
>> Regards,
>> Jordan
>>
>> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss
>> <gr...@namhost.com.invalid> <ma...@namhost.com.invalid> wrote:
>>
>>> Hi Guys
>>>
>>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>>> not successful and gave an error stating that it cannot be deleted because
>>> there are active VMs associated with it. However, since I did this, I can
>>> now no longer deploy new VMs using said template. Creating a VM now fails
>>> since I did this, see attached screenshot.
>>>
>>> And this is the management log entry:
>>>
>>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>>> (logid:4c740aa8) Rethrow exception
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>>> 2783
>>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>>> cmdInfo:
>>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>>> null}, job origin:2783
>>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>>> com.cloud.dc.DataCenter; id=16
>>>
>>> This seems to ONLY happen with this one template that I deleted, I can
>>> deploy VMs successfully from other templates but this one that was
>>> "deleted", even though it was NOT deleted. So it seems the template broke
>>> somehow because now no volume is created for new VMs when using this
>>> template. I am using 4.17.1.0. Is this a known issue?
>>> --
>>> Regards / Groete
>>>
>>> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>>
>>> *e:*granwille@namhost.com
>>> *m:* +264 81 323 1260 <+264813231260>
>>> *w:*www.namhost.com <http://www.namhost.com>
>>>
>>> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost>
>>> <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/>
>>> <https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos>
>>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>>
>>>
>>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>>
>>> Namhost Internet Services (Pty) Ltd,
>>>
>>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>>
>>>
>>>
>>> The content of this message is confidential. If you have received it by
>>> mistake, please inform us by email reply and then delete the message. It is
>>> forbidden to copy, forward, or in any way reveal the contents of this
>>> message to anyone without our explicit consent. The integrity and security
>>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>>> will not be held liable for any damage caused by the message. For our full
>>> privacy policy and disclaimers, please go to
>>> https://www.namhost.com/privacy-policy
>>>
>>> [image: Powered by AdSigner]
>>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <tel:+264813231260>
> *w:* www.namhost.com <https://www.namhost.com/>
>
> <https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received
> it by mistake, please inform us by email reply and then delete the
> message. It is forbidden to copy, forward, or in any way reveal
> the contents of this message to anyone without our explicit
> consent. The integrity and security of this email cannot be
> guaranteed over the Internet. Therefore, the sender will not be
> held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to
> https://www.namhost.com/privacy-policy
>
> Powered by AdSigner
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by jordan j <yo...@gmail.com>.
Here is the issue, ACS cannot find the template volume to copy -
d2c4feca-da5c-44b3-8ecc-40a01fd6c243
2022-12-20 07:59:52,453 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
(logid:4c740aa8) Seq 36-1710804908447415280: Sending { Cmd , MgmtId:
66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","uuid":"d2c4feca-da5c-44b3-8ecc-40a01fd6c243","id":"212","format":"QCOW2","accountId":"2","checksum":"{SHA-512}fbb1808d2823bfad0ac8c17ddc1038871b4678ed326b91ac4949376dd9823da0153faa031ad00ed3ad9772d78982d8fd2bdf3816b1f2d894d10fa5a72fb5d638","hvm":"true","displayText":"Almalinux
8 cPanel Template","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","size":"(60,00
GB) 64424509440","hypervisorType":"KVM","bootable":"false","uniqueName":"5225a781-17b1-4cc6-b112-0e110e2ca4e5","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"88695a0f-ac1e-42b8-8eb8-61b258e9e064","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3a808086-b3bd-3dda-b27d-18ab9f0059f4","name":"Prime","id":"25","poolType":"NetworkFilesystem","host":"192.168.50.2","path":"/mnt/dataprime","port":"2049","url":"NetworkFilesystem://192.168.50.2/mnt/dataprime/?ROLE=Primary&STOREUUID=3a808086-b3bd-3dda-b27d-18ab9f0059f4","isManaged":"false"}},"name":"ROOT-100","size":"(60,00
GB) 64424509440","volumeId":"104","vmName":"i-2-100-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"25","id":"104","deviceId":"0","cacheMode":"NONE","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"0","bypassHostMaintenance":"false"}}]
}
2022-12-20 08:02:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-6616ece4) (logid:5c6d1e6b) Task (job-2783) has been
pending for 141 seconds
2022-12-20 08:03:11,515 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-60f0ac1b) (logid:5e5388e3) Task (job-2783) has been
pending for 201 seconds
2022-12-20 08:04:11,514 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-0:ctx-2e7c936a) (logid:a3027c48) Task (job-2783) has been
pending for 261 seconds
2022-12-20 08:04:55,237 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
(logid:4c740aa8) Seq 36-1710804908447415280: Received: { Ans: ,
MgmtId: 66988330791812, via: 36(athena03.namhost.com), Ver: v1, Flags:
110, { CopyCmdAnswer } }
2022-12-20 08:04:55,241 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
(logid:4c740aa8) Unable to create
Vol[104|vm=100|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
Can't find volume:d2c4feca-da5c-44b3-8ecc-40a01fd6c243
If other templates can be used from the same SR this means it is accessible
but the template file is not there anymore.
To look for it you can login on the secondary storage and look at a path
similar to this <mount dir>/../templates.
inside should be 1 folder for each template, it will include XML meta file
and volume file. The volume, if exists should be named by the ID
d2c4feca-da5c-44b3-8ecc-40a01fd6c243.
if no such is present than indeed it is deleted.
Regards,
Jordan
On Tue, Dec 20, 2022 at 7:23 AM Granwille Strauss <gr...@namhost.com>
wrote:
> Hi Jordan
>
> Please see log attached. Can you confirm what's the best way to confirm if
> a template exists on secondary storage?
> On 12/20/22 08:15, jordan j wrote:
>
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss<gr...@namhost.com.invalid> <gr...@namhost.com.invalid> wrote:
>
>
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted because
> there are active VMs associated with it. However, since I did this, I can
> now no longer deploy new VMs using said template. Creating a VM now fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete
> <https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
> <https://www.facebook.com/namhost> <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to
> https://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
Re: Cannot create VM from Template Error
Posted by Granwille Strauss <gr...@namhost.com.INVALID>.
Hi Jordan
Please see log attached. Can you confirm what's the best way to confirm
if a template exists on secondary storage?
On 12/20/22 08:15, jordan j wrote:
> Hey Granwille Strauss,
>
> Can you upload a full log somewhere?
> This error usually means there is another underlying issue even though it
> says Network/interface scope.
> Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> Can you also verify the template file is intact on the secondary server?
>
> Regards,
> Jordan
>
> On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss
> <gr...@namhost.com.invalid> wrote:
>
>> Hi Guys
>>
>> I accidentality "deleted" a wrong template. Fortunately, the delete was
>> not successful and gave an error stating that it cannot be deleted because
>> there are active VMs associated with it. However, since I did this, I can
>> now no longer deploy new VMs using said template. Creating a VM now fails
>> since I did this, see attached screenshot.
>>
>> And this is the management log entry:
>>
>> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
>> (logid:4c740aa8) Rethrow exception
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
>> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
>> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
>> 2783
>> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
>> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
>> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
>> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
>> cmdInfo:
>> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
>> null}, job origin:2783
>> com.cloud.exception.InsufficientServerCapacityException: Unable to create
>> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
>> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
>> com.cloud.dc.DataCenter; id=16
>>
>> This seems to ONLY happen with this one template that I deleted, I can
>> deploy VMs successfully from other templates but this one that was
>> "deleted", even though it was NOT deleted. So it seems the template broke
>> somehow because now no volume is created for new VMs when using this
>> template. I am using 4.17.1.0. Is this a known issue?
>> --
>> Regards / Groete
>>
>> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>>
>> *e:*granwille@namhost.com
>> *m:* +264 81 323 1260 <+264813231260>
>> *w:*www.namhost.com
>>
>> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
>> <https://www.instagram.com/namhostinternetservices/>
>> <https://www.linkedin.com/company/namhos>
>> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>>
>>
>> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>>
>> Namhost Internet Services (Pty) Ltd,
>>
>> 24 Black Eagle Rd, Hermanus, 7210, RSA
>>
>>
>>
>> The content of this message is confidential. If you have received it by
>> mistake, please inform us by email reply and then delete the message. It is
>> forbidden to copy, forward, or in any way reveal the contents of this
>> message to anyone without our explicit consent. The integrity and security
>> of this email cannot be guaranteed over the Internet. Therefore, the sender
>> will not be held liable for any damage caused by the message. For our full
>> privacy policy and disclaimers, please go to
>> https://www.namhost.com/privacy-policy
>>
>> [image: Powered by AdSigner]
>> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
Re: Cannot create VM from Template Error
Posted by jordan j <yo...@gmail.com>.
Hey Granwille Strauss,
Can you upload a full log somewhere?
This error usually means there is another underlying issue even though it
says Network/interface scope.
Deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
"e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
com.cloud.dc.DataCenter; id=16
Can you also verify the template file is intact on the secondary server?
Regards,
Jordan
On Tue, Dec 20, 2022 at 8:09 AM Granwille Strauss
<gr...@namhost.com.invalid> wrote:
> Hi Guys
>
> I accidentality "deleted" a wrong template. Fortunately, the delete was
> not successful and gave an error stating that it cannot be deleted because
> there are active VMs associated with it. However, since I did this, I can
> now no longer deploy new VMs using said template. Creating a VM now fails
> since I did this, see attached screenshot.
>
> And this is the management log entry:
>
> 2022-12-20 08:04:55,618 INFO [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784 ctx-5325e481)
> (logid:4c740aa8) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
> 2022-12-20 08:04:55,618 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 100, job origin:
> 2783
> 2022-12-20 08:04:55,618 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-5298589d job-2783/job-2784) (logid:4c740aa8)
> Unable to complete AsyncJobVO: {id:2784, userId: 2, accountId: 2,
> instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart,
> cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAEHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAA1zcQB-AAgAAAAAAAAAJHBwcHNxAH4ACAAAAAAAAAAPcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 66988330791812, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: Tue Dec 20 07:59:49 SAST 2022, removed:
> null}, job origin:2783
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a deployment for VM instance {id: "100", name: "i-2-100-VM", uuid:
> "e0611ff1-cab5-48d0-ac4e-09fe0f034b19", type="User"}Scope=interface
> com.cloud.dc.DataCenter; id=16
>
> This seems to ONLY happen with this one template that I deleted, I can
> deploy VMs successfully from other templates but this one that was
> "deleted", even though it was NOT deleted. So it seems the template broke
> somehow because now no volume is created for new VMs when using this
> template. I am using 4.17.1.0. Is this a known issue?
> --
> Regards / Groete
>
> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin
>
> *e:* granwille@namhost.com
> *m:* +264 81 323 1260 <+264813231260>
> *w:* www.namhost.com
>
> <https://www.facebook.com/namhost> <https://twitter.com/namhost>
> <https://www.instagram.com/namhostinternetservices/>
> <https://www.linkedin.com/company/namhos>
> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
>
> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
>
> The content of this message is confidential. If you have received it by
> mistake, please inform us by email reply and then delete the message. It is
> forbidden to copy, forward, or in any way reveal the contents of this
> message to anyone without our explicit consent. The integrity and security
> of this email cannot be guaranteed over the Internet. Therefore, the sender
> will not be held liable for any damage caused by the message. For our full
> privacy policy and disclaimers, please go to
> https://www.namhost.com/privacy-policy
>
> [image: Powered by AdSigner]
> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>