You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "vignesh-goutham (via GitHub)" <gi...@apache.org> on 2023/04/12 15:31:53 UTC
[GitHub] [cloudstack] vignesh-goutham opened a new issue, #7428: Destroy vm API errors with failure to delete ConfigDrive
vignesh-goutham opened a new issue, #7428:
URL: https://github.com/apache/cloudstack/issues/7428
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
API
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main branch.
-->
~~~
4.16
~~~
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
Host and Guest running RHEL 8.x on KVM hypervisor
##### SUMMARY
[CAPC](https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack) creates and deletes VM depending on user requests and configuration. On occasions, the delete API calls is not successful. The API returns success, but the VM does not get deleted. CAPC issues [Delete VM call](https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack/blob/5f8ef8020086bd4f8c7136e78a06311a848984fa/pkg/cloud/instance.go#L296) and waits for the VM to be deleted. On debugging the issue more, cloudstack logs show exceptions in deleting config drive for the vm. Cloudstack logs below.
```
2023-01-14 04:35:34,842 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-118:ctx-8528bb01 job-55****/job-55**** ctx-f09653a3) (logid:******) VM is already stopped: VM instance {id: "5436", name: "i-212-5436-VM", uuid: "****REDACTED******", type="User"}
2023-01-14 00:01:23,150 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-117:ctx-bb191bb8 job-54****) (logid:*****) Unexpected exception while executing org.apache.cloudstack.api.command.user.vm.DestroyVMCmd com.cloud.utils.exception.CloudRuntimeException: Unable to get an answer to handle config drive deletion for vm: i-212-5436-VM on host: 804 at com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIsoOnHostCache(ConfigDriveNetworkElement.java:577) at com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIso(ConfigDriveNetworkElement.java:665) at com.cloud.network.element.ConfigDriveNetworkElement.postStateTransitionEvent(ConfigDriveNetworkElement.java:322) at com.cloud.network.element.ConfigDriveNetworkElement.postStateTransitionEvent(ConfigDriveNetworkElement.java:90)
2023-01-14 04:37:45,034 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-10:ctx-32e2ec2b job-55***) (logid:*****) Complete async job-55***, jobStatus: FAILED, resultCode: 530, result: org.apache.cloud
stack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Unable to destroy VM instance {id: "5436", name: "i-212-5436-VM", uuid: "****REDACTED******", type="User"} due to [Unable to transition to a new state from Expunging via DestroyRequested]."}
2023-01-14 04:37:44,952 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:*****) Request:Seq 804-6595240179307385555: { Cmd , MgmtId: ***********, via: 804, Ver: v1, Flags: 100011, [{"com.cloud.a gent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-212-5436-VM.iso","create":"false","useHostCacheOnUnsupportedPool":"false","preferHostCache":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-01-14 04:37:44,952 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:*****) Processing command: com.cloud.agent.api.HandleConfigDriveIsoCommand
2023-01-14 04:37:44,952 DEBUG [kvm.resource.ComputingResource] (agentRequest-Handler-1:null) (logid:********) ComputingResource routing command to LibvirtComputingResource
2023-01-14 04:37:44,952 DEBUG [resource.wrapper.LibvirtHandleConfigDriveCommandWrapper] (agentRequest-Handler-1:null) (logid:*****) Deleting config drive: configdrive/i-212-5436-VM.iso
2023-01-14 04:37:44,953 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:******) Seq 804-6595240179307385555: { Ans: , MgmtId: ************, via: 804, Ver: v1, Flags: 10, [{"com.cloud.agent.api.HandleConfigDriveIsoAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-01-14 04:37:48,212 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:*****) Request:Seq 804-6595240179307385556: { Cmd , MgmtId: ********, via: 804, Ver: v1, Flags: 100011, [{"com.cloud.a gent.api.HandleConfigDriveIsoCommand":{"isoFile":"configdrive/i-212-5436-VM.iso","create":"false","useHostCacheOnUnsupportedPool":"false","preferHostCache":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-01-14 04:37:48,212 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:*****) Processing command: com.cloud.agent.api.HandleConfigDriveIsoCommand
2023-01-14 04:37:48,212 DEBUG [kvm.resource.ComputingResource] (agentRequest-Handler-2:null) (logid:*******) ComputingResource routing command to LibvirtComputingResource
2023-01-14 04:37:48,212 DEBUG [resource.wrapper.LibvirtHandleConfigDriveCommandWrapper] (agentRequest-Handler-2:null) (logid:*****) Deleting config drive: configdrive/i-212-5436-VM.iso
2023-01-14 04:37:48,213 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:*******) Seq 804-6595240179307385556: { Ans: , MgmtId: ********, via: 804, Ver: v1, Flags: 10, [{"com.cloud.agent.api.HandleConfigDriveIsoAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
```
If I request a delete VM on the UI, Cloudstack proceeds to delete the VM properly. API call used by CAPC is [destroyVirtualMachine](https://cloudstack.apache.org/api/apidocs-4.18/apis/destroyVirtualMachine.html).
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
1 out of every 30-40 destroy VM API hits this issue.
~~~
<!-- You can also paste gist.github.com links for larger files -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] boring-cyborg[bot] commented on issue #7428: Destroy vm API errors with failure to delete ConfigDrive
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #7428:
URL: https://github.com/apache/cloudstack/issues/7428#issuecomment-1505481863
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on issue #7428: Destroy vm API errors with failure to delete ConfigDrive
Posted by "rohityadavcloud (via GitHub)" <gi...@apache.org>.
rohityadavcloud commented on issue #7428:
URL: https://github.com/apache/cloudstack/issues/7428#issuecomment-1602223310
Related improvement - as part of this https://github.com/kubernetes-sigs/image-builder/pull/1202 was submitted as capc cannot deploy the default images on networks with ConfigDrive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on issue #7428: Destroy vm API errors with failure to delete ConfigDrive
Posted by "rohityadavcloud (via GitHub)" <gi...@apache.org>.
rohityadavcloud commented on issue #7428:
URL: https://github.com/apache/cloudstack/issues/7428#issuecomment-1603907438
Fixed in https://github.com/apache/cloudstack/pull/7584
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on issue #7428: Destroy vm API errors with failure to delete ConfigDrive
Posted by "rohityadavcloud (via GitHub)" <gi...@apache.org>.
rohityadavcloud commented on issue #7428:
URL: https://github.com/apache/cloudstack/issues/7428#issuecomment-1566840142
@vishesh92 I read from the ticket - it's 4.16, likely 4.16.1.0. You had pinged me on slack about https://github.com/apache/cloudstack/pull/6234 it's possible this fix isn't in 4.16.1.0 or 4.16.2.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] vishesh92 commented on issue #7428: Destroy vm API errors with failure to delete ConfigDrive
Posted by "vishesh92 (via GitHub)" <gi...@apache.org>.
vishesh92 commented on issue #7428:
URL: https://github.com/apache/cloudstack/issues/7428#issuecomment-1566698030
@vignesh-goutham What is the version of cloudstack you are using? Please specify minor version as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud closed issue #7428: Destroy vm API errors with failure to delete ConfigDrive
Posted by "rohityadavcloud (via GitHub)" <gi...@apache.org>.
rohityadavcloud closed issue #7428: Destroy vm API errors with failure to delete ConfigDrive
URL: https://github.com/apache/cloudstack/issues/7428
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org