You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2022/03/26 21:56:28 UTC
[GitHub] [cloudstack] rohityadavcloud opened a new issue #6167: Fails to expunge VM post upgrade to 4.16.1.
rohityadavcloud opened a new issue #6167:
URL: https://github.com/apache/cloudstack/issues/6167
I'm not sure if this is an env issue or something with 4.16.1.0, post upgrade to 4.16.1.0 I created a new VM and then destroyed it. However, I was able to destroy the old systemvms and router without any issue.
On destroy, it fails (both with or without expunge option):
```
2022-03-26 21:53:03,574 DEBUG [c.c.a.ApiServlet] (qtp603305436-268:ctx-ac47f422) (logid:d55248c9) ===START=== 192.168.1.101 -- GET jobId=08a43e6a-d34f-4f4e-b5b6-52aa098df2e6&command=queryAsyncJobResult&response=json
2022-03-26 21:53:03,601 DEBUG [c.c.a.ApiServer] (qtp603305436-268:ctx-ac47f422 ctx-2a3a17fb) (logid:d55248c9) CIDRs from which account 'Acct[cff9c6d0-3b42-11ec-b17a-dca632aeaa71-admin] -- Account {"id": 2, "name": "admin", "uuid": "cff9c6d0-3b42-11ec-b17a-dca632aeaa71"}' is allowed to perform API calls: 192.168.1.0/24,10.6.0.1/24
2022-03-26 21:53:03,645 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) Checking if there are any ongoing snapshots on the ROOT volumes associated with VM with ID 140
2022-03-26 21:53:03,653 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) Found 1 no. of volumes of type ROOT for vm with VM ID 140
2022-03-26 21:53:03,653 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) Checking status of snapshots for Volume with Volume Id: 142
2022-03-26 21:53:03,660 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) The count of ongoing Snapshots for VM with ID 140 and disk type ROOT is 0
2022-03-26 21:53:03,661 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) Found no ongoing snapshots on volume of type ROOT, for the vm with id 140
2022-03-26 21:53:03,723 DEBUG [c.c.a.ApiServlet] (qtp603305436-268:ctx-ac47f422 ctx-2a3a17fb) (logid:d55248c9) ===END=== 192.168.1.101 -- GET jobId=08a43e6a-d34f-4f4e-b5b6-52aa098df2e6&command=queryAsyncJobResult&response=json
2022-03-26 21:53:03,789 DEBUG [c.c.u.d.T.Transaction] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) Rolling back the transaction: Time = 8 Name = API-Job-Executor-6; called by -TransactionLegacy.rollback:888-TransactionLegacy.removeUpTo:831-TransactionLegacy.close:655-Transaction.execute:38-Transaction.execute:47-AsyncJobManagerImpl.submitAsyncJob:237-VirtualMachineManagerImpl.stopVmThroughJobQueue:5365-VirtualMachineManagerImpl.advanceStop:1872-VirtualMachineManagerImpl.stopForced:1569-VMEntityManagerImpl.stopvirtualmachineforced:275-VirtualMachineEntityImpl.stopForced:224-UserVmManagerImpl.stopVirtualMachine:5046
2022-03-26 21:53:03,811 WARN [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108 ctx-e46fec56) (logid:08a43e6a) Unable to schedule async job for command com.cloud.vm.VmWorkStop, unexpected exception.
javax.persistence.EntityExistsException: Entity already exists
at com.cloud.utils.db.GenericDaoBase.handleEntityExistsException(GenericDaoBase.java:859)
at com.cloud.utils.db.GenericDaoBase.persist(GenericDaoBase.java:1461)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.doInTransaction(AsyncJobManagerImpl.java:241)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.doInTransaction(AsyncJobManagerImpl.java:237)
at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50)
at com.cloud.utils.db.Transaction.execute(Transaction.java:40)
at com.cloud.utils.db.Transaction.execute(Transaction.java:47)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl.submitAsyncJob(AsyncJobManagerImpl.java:237)
at com.cloud.vm.VirtualMachineManagerImpl.stopVmThroughJobQueue(VirtualMachineManagerImpl.java:5365)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1872)
at com.cloud.vm.VirtualMachineManagerImpl.stopForced(VirtualMachineManagerImpl.java:1569)
at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.stopvirtualmachineforced(VMEntityManagerImpl.java:275)
at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.stopForced(VirtualMachineEntityImpl.java:224)
at com.cloud.vm.UserVmManagerImpl.stopVirtualMachine(UserVmManagerImpl.java:5046)
at com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:3209)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy127.destroyVm(Unknown Source)
at org.apache.cloudstack.api.command.user.vm.DestroyVMCmd.execute(DestroyVMCmd.java:137)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry 'Prepare-140' for key 'vm_work_job.vm_work_job_step_and_objects'
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:117)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
... 51 more
2022-03-26 21:53:03,860 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-6:ctx-f1c0e1e9 job-1108) (logid:08a43e6a) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin
com.cloud.utils.exception.CloudRuntimeException: Unable to schedule async job for command com.cloud.vm.VmWorkStop, unexpected exception.
```
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
Async job
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main branch.
-->
~~~
4.16.1.0
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on issue #6167: Fails to expunge VM post upgrade to 4.16.1.0
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on issue #6167:
URL: https://github.com/apache/cloudstack/issues/6167#issuecomment-1079783190
Something went wrong in the upgrade because of which this line with secondary_object didn't work:
https://github.com/apache/cloudstack/blob/main/engine/schema/src/main/resources/META-INF/db/schema-41600to41610.sql#L22
The issue was incorrect unique key, fixing that fixed this issue:
```
| vm_work_job | CREATE TABLE `vm_work_job` (
`id` bigint unsigned NOT NULL,
`step` char(32) NOT NULL COMMENT 'state',
`vm_type` char(32) NOT NULL COMMENT 'type of vm',
`vm_instance_id` bigint unsigned NOT NULL COMMENT 'vm instance',
`secondary_object` char(100) DEFAULT NULL COMMENT 'any additional item that must be checked during queueing',
PRIMARY KEY (`id`),
UNIQUE KEY `vm_work_job_step_and_objects` (`step`,`vm_instance_id`),
KEY `fk_vm_work_job__instance_id` (`vm_instance_id`),
KEY `i_vm_work_job__vm` (`vm_type`,`vm_instance_id`),
KEY `i_vm_work_job__step` (`step`),
CONSTRAINT `fk_vm_work_job__instance_id` FOREIGN KEY (`vm_instance_id`) REFERENCES `vm_instance` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud closed issue #6167: Fails to expunge VM post upgrade to 4.16.1.0
Posted by GitBox <gi...@apache.org>.
rohityadavcloud closed issue #6167:
URL: https://github.com/apache/cloudstack/issues/6167
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org