You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2023/01/05 03:27:05 UTC

[GitHub] [cloudstack] fangyi1008 opened a new issue, #7053: cant not instance

fangyi1008 opened a new issue, #7053:
URL: https://github.com/apache/cloudstack/issues/7053

   2023-01-05 10:36:33,001 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Destination Host to deploy the VM is specified, specifying a deployment plan to deploy the VM
   2023-01-05 10:36:33,004 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Host: 1 has cpu capability (cpu:128, speed:2000000) to support requested CPU: 1 and requested speed: 1000
   2023-01-05 10:36:33,004 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Checking if host: 1 has enough capacity for requested CPU: 1000 and requested RAM: (1.00 GB) 1073741824 , cpuOverprovisioningFactor: 1.0
   2023-01-05 10:36:33,005 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Hosts's actual total CPU: 256000000 and CPU after applying overprovisioning: 256000000
   2023-01-05 10:36:33,005 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) considerReservedCapacity isfalse , not considering reserved capacity for calculating free capacity
   2023-01-05 10:36:33,005 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Free CPU: 255999000 , Requested CPU: 1000
   2023-01-05 10:36:33,006 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Free RAM: (501.02 GB) 537961562112 , Requested RAM: (1.00 GB) 1073741824
   2023-01-05 10:36:33,006 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Host has enough CPU and RAM available
   2023-01-05 10:36:33,006 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) STATS: Can alloc CPU from host: 1, used: 1000, reserved: 0, actual total: 256000000, total with overprovisioning: 256000000; requested cpu:
   1000,alloc_from_last_host?:false ,considerReservedCapacity?: false2023-01-05 10:36:33,006 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) STATS: Can alloc MEM from host: 1, used: (1.50 GB) 1610612736, reserved: (0 bytes) 0, total: (502.52 GB) 539572174848; requested mem: (1.00
    GB) 1073741824, alloc_from_last_host?: false , considerReservedCapacity?: false2023-01-05 10:36:33,009 DEBUG [c.c.a.ApiServlet] (qtp1747352992-285:ctx-38d5a57c) (logid:e0539ea9) ===START===  192.168.0.102 -- GET  jobId=8270895e-bb89-41f6-8f67-3c2f33a4b30d&command=queryAsyncJobResult&response=json
   2023-01-05 10:36:33,014 DEBUG [c.c.a.ApiServer] (qtp1747352992-285:ctx-38d5a57c ctx-b4ed1e0d) (logid:e0539ea9) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d"
   }' is allowed to perform API calls: 0.0.0.0/0,::/02023-01-05 10:36:33,015 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Adding pods to avoid lists for non-explicit VM deployment: []
   2023-01-05 10:36:33,015 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Adding clusters to avoid lists for non-explicit VM deployment: []
   2023-01-05 10:36:33,015 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Adding hosts to avoid lists for non-explicit VM deployment: []
   2023-01-05 10:36:33,015 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) DeploymentPlanner allocation algorithm: null
   2023-01-05 10:36:33,015 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 1000, requested ram: (1.00 GB) 1073741824
   2023-01-05 10:36:33,015 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Is ROOT volume READY (pool already allocated)?: No
   2023-01-05 10:36:33,019 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) DeploymentPlan has host_id specified, choosing this host and making no checks on this host: 1
   2023-01-05 10:36:33,020 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   2023-01-05 10:36:33,022 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Checking suitable pools for volume (Id, Type): (70,ROOT)
   2023-01-05 10:36:33,022 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) We need to allocate new storagepool for this volume
   2023-01-05 10:36:33,023 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Calling StoragePoolAllocators to find suitable pools
   2023-01-05 10:36:33,024 DEBUG [c.c.a.ApiServlet] (qtp1747352992-285:ctx-38d5a57c ctx-b4ed1e0d) (logid:e0539ea9) ===END===  192.168.0.102 -- GET  jobId=8270895e-bb89-41f6-8f67-3c2f33a4b30d&command=queryAsyncJobResult&response=json
   2023-01-05 10:36:33,027 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   2023-01-05 10:36:33,027 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) No suitable pools found
   2023-01-05 10:36:33,027 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Cannot deploy to specified host, returning.
   2023-01-05 10:36:33,028 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Destroying vm VM instance {id: "90", name: "i-2-90-VM", uuid: "27615bc8-9d9f-4469-a2d7-215de3bd6a31", type="User"} as it failed to create on 
   Host with Id:12023-01-05 10:36:33,045 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) VM instance {id: "90", name: "i-2-90-VM", uuid: "27615bc8-9d9f-4469-a2d7-215de3bd6a31", type="User"} state transited from [Stopped] to [Err
   or] with event [OperationFailedToError]. VM's original host: null, new host: null, host before state transition: null2023-01-05 10:36:33,059 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Updating resource Type = volume count for Account = 2 Operation = decreasing Amount = 1
   2023-01-05 10:36:33,069 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Updating resource Type = primary_storage count for Account = 2 Operation = decreasing Amount = (20.00 GB) 21474836480
   2023-01-05 10:36:33,085 WARN  [c.c.a.AlertManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) alertType=[8] dataCenterId=[1] podId=[null] clusterId=[null] message=[Failed to deploy Vm with Id: 90, on Host with Id: 1].
   2023-01-05 10:36:33,091 WARN  [c.c.a.AlertManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Failed to deploy Vm with Id: 90, on Host wit
   h Id: 1] and content [Failed to deploy Vm with Id: 90, on Host with Id: 1].2023-01-05 10:36:33,092 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Updating resource Type = user_vm count for Account = 2 Operation = decreasing Amount = 1
   2023-01-05 10:36:33,100 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Updating resource Type = cpu count for Account = 2 Operation = decreasing Amount = 1
   2023-01-05 10:36:33,105 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) Updating resource Type = memory count for Account = 2 Operation = decreasing Amount = 1024
   2023-01-05 10:36:33,118 INFO  [o.a.c.a.c.u.v.DeployVMCmd] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) com.cloud.exception.InsufficientServerCapacityException: No destination found for a deployment for VM instance {id: "90", name: "i-2-90-VM"
   , uuid: "27615bc8-9d9f-4469-a2d7-215de3bd6a31", type="User"}Scope=interface com.cloud.dc.DataCenter; id=12023-01-05 10:36:33,120 INFO  [o.a.c.a.c.u.v.DeployVMCmd] (API-Job-Executor-18:ctx-41c893ef job-305 ctx-e4d8c684) (logid:8270895e) No destination found for a deployment for VM instance {id: "90", name: "i-2-90-VM", uuid: "27615bc8-9d9f-4469-a2d7-215de3bd6a31", type="Use
   r"}com.cloud.exception.InsufficientServerCapacityException: No destination found for a deployment for VM instance {id: "90", name: "i-2-90-VM", uuid: "27615bc8-9d9f-4469-a2d7-215de3bd6a31", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
   	at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.reserveVirtualMachine(VMEntityManagerImpl.java:225)
   	at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.reserve(VirtualMachineEntityImpl.java:202)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5310)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5161)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4788)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4777)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   	at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
   	at com.sun.proxy.$Proxy180.startVirtualMachine(Unknown Source)
   	at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:714)
   	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163)
   	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:106)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   2023-01-05 10:36:33,129 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-18:ctx-41c893ef job-305) (logid:8270895e) Remove job-305 from job monitoring
   2023-01-05 10:36:33,353 DEBUG [c.c.a.ApiServlet] (qtp1747352992-514:ctx-bc1de5cb) (logid:9774c341) ===START===  192.168.0.102 -- GET  resourceids=8f55ef53-d2bf-4906-b115-fe527160f7aa&resourcetype=template&command=listResourceIcon&response=json
   2023-01-05 10:36:33,354 DEBUG [c.c.a.ApiServlet] (qtp1747352992-458:ctx-9e8902ce) (logid:39d7b02f) ===START===  192.168.0.102 -- GET  listall=true&page=1&pagesize=20&showIcon=true&command=listVirtualMachinesMetrics&response=json
   2023-01-05 10:36:33,357 DEBUG [c.c.a.ApiServer] (qtp1747352992-514:ctx-bc1de5cb ctx-1c2851ae) (logid:9774c341) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d"
   }' is allowed to perform API calls: 0.0.0.0/0,::/02023-01-05 10:36:33,358 DEBUG [c.c.a.ApiServer] (qtp1747352992-458:ctx-9e8902ce ctx-7f77c6f8) (logid:39d7b02f) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d"
   }' is allowed to perform API calls: 0.0.0.0/0,::/0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371990878

   > Does your service offering use local storage?
   I know how to solve it。thank you


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371945664

   > 没有合适的目标主机来部署您的实例。请检查是否有足够的可用容量来部署您指定大小的实例。
   ![image](https://user-images.githubusercontent.com/89246132/210740919-ff8d155e-2545-4d14-86cc-5d9221a131a4.png)
   The vols queried here is an empty array,The pool_id in the volumes table is also null,
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371957668

   > @fangyi1008 您使用本地存储还是共享存储?
   > 
   > 日志显示没有适合该卷的存储池。
   > 
   > ```
   > 2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   > 2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   > 2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   > 2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   > 2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   > 2023-01-05 10:36:33,027 No suitable pools found
   > 2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   > ```
   The primary storage is local storage, and the secondary storage uses nfs。
   In the global configuration, system.vm.use.local.storage is set to true
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371987731

   > 
   
   
   
   > > > @fangyi1008 您使用本地存储还是共享存储?
   > > > 日志显示没有适合该卷的存储池。
   > > > ```
   > > > 2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   > > > 2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   > > > 2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   > > > 2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   > > > 2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   > > > 2023-01-05 10:36:33,027 No suitable pools found
   > > > 2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   > > > ```
   > > 
   > > 
   > > The primary storage is local storage, and the secondary storage uses nfs。 In the global configuration, system.vm.use.local.storage is set to true
   > 
   > @fangyi1008 what's the storage type of service offering (used by the vm instance) ? it should be "local" as well.
   
   <img width="627" alt="f4f1e13f3575c7d38ca5da1e454c6cb" src="https://user-images.githubusercontent.com/89246132/210748722-bc80f1ac-7253-4d3c-ac89-9b3d88e2f42c.png">
   The zone is created as follows
   
   ![image](https://user-images.githubusercontent.com/89246132/210748997-b794cd3e-1839-443b-916a-bffa093379d8.png)
   Uploaded iso used for instance creation


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] weizhouapache commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371954714

   @fangyi1008 
   do you use local storage or shared storage ?
   
   The logs show that there is no suitable storage pool for the volume.
   ```
   2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   2023-01-05 10:36:33,027 No suitable pools found
   2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371994078

   > > > @fangyi1008 您使用本地存储还是共享存储?
   > > > 日志显示没有适合该卷的存储池。
   > > > ```
   > > > 2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   > > > 2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   > > > 2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   > > > 2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   > > > 2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   > > > 2023-01-05 10:36:33,027 No suitable pools found
   > > > 2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   > > > ```
   > > 
   > > 
   > > The primary storage is local storage, and the secondary storage uses nfs。 In the global configuration, system.vm.use.local.storage is set to true
   > 
   > @fangyi1008 what's the storage type of service offering (used by the vm instance) ? it should be "local" as well.
   Another problem arises。
   Unable to start a VM [fdb11a9c-b45a-4a32-b6f9-c322a2358444] due to [Unable to create a deployment for VM instance {id: "98", name: "i-2-98-VM", uuid: "fdb11a9c-b45a-4a32-b6f9-c322a2358444", type="User"}].
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] weizhouapache commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371980503

   > > @fangyi1008 您使用本地存储还是共享存储?
   > > 日志显示没有适合该卷的存储池。
   > > ```
   > > 2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   > > 2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   > > 2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   > > 2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   > > 2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   > > 2023-01-05 10:36:33,027 No suitable pools found
   > > 2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   > > ```
   > 
   > The primary storage is local storage, and the secondary storage uses nfs。 In the global configuration, system.vm.use.local.storage is set to true
   
   @fangyi1008 
   what's the storage type of service offering (used by the vm instance) ?
   it should be "local" as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] weizhouapache commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1372030371

   > > can you share more logs, please ?
   > > management log:
   > 
   > ## 2023-01-05 17:52:11,528 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 101, job origin: 446
   > 2023-01-05 17:52:11,528 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) Unable to complete AsyncJobVO: {id:447, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
   > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 26183377
 3422727, completeMsid: null, lastUpdated: null, lastPolled: null, created: Thu Jan 05 17:51:56 CST 2023, removed: null}, job origin:446com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
   > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1140)
   > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   > at jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source)
   > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   > at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
   > at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
   > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
   > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   > at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   > at java.base/java.lang.Thread.run(Thread.java:829)
   > 2023-01-05 17:52:11,541 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) Remove job-447 from job monitoring
   > 2023-01-05 17:52:11,554 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Destroying vm VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"} as it failed to create
   > on Host with Id:12023-01-05 17:52:11,571 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"} state transited from [Stopped] to [
   > Error] with event [OperationFailedToError]. VM's original host: null, new host: null, host before state transition: null2023-01-05 17:52:11,586 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = volume count for Account = 2 Operation = decreasing Amount = 1
   > 2023-01-05 17:52:11,595 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = primary_storage count for Account = 2 Operation = decreasing Amount = (20.00 GB) 21474836480
   > 2023-01-05 17:52:11,614 WARN [c.c.a.AlertManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) alertType=[8] dataCenterId=[1] podId=[1] clusterId=[null] message=[Failed to deploy Vm with Id: 101, on Host with Id: 1].
   > 2023-01-05 17:52:11,621 WARN [c.c.a.AlertManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Failed to deploy Vm with Id: 101, on Host w
   > ith Id: 1] and content [Failed to deploy Vm with Id: 101, on Host with Id: 1].2023-01-05 17:52:11,622 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = user_vm count for Account = 2 Operation = decreasing Amount = 1
   > 2023-01-05 17:52:11,630 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = cpu count for Account = 2 Operation = decreasing Amount = 1
   > 2023-01-05 17:52:11,637 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = memory count for Account = 2 Operation = decreasing Amount = 512
   > 2023-01-05 17:52:11,653 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-117:ctx-f896976b job-446) (logid:564609bc) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin
   > com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM [60093c58-bba6-470e-a958-0d30e5507628] due to [Unable to create a deployment for VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}].
   > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:803)
   > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
   > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:214)
   > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5311)
   > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5161)
   > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4788)
   > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4777)
   > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   > at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
   > at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
   > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   > at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
   > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   > at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
   > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   > at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
   > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   > at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
   > at com.sun.proxy.$Proxy180.startVirtualMachine(Unknown Source)
   > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:714)
   > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163)
   > at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:106)
   > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   > at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   > at java.base/java.lang.Thread.run(Thread.java:829)
   > Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
   > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1140)
   > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   > at jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source)
   > ... 17 more
   > 2023-01-05 17:52:11,664 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-117:ctx-f896976b job-446) (logid:564609bc) Remove job-446 from job monitoring
   > 2023-01-05 17:52:11,768 DEBUG [c.c.a.ApiServlet] (qtp1747352992-8705:ctx-d2189185) (logid:b61b54a5) ===START=== 192.168.0.102 -- GET jobId=564609bc-a5e0-43eb-a1a9-45f5513d3ee0&command=queryAsyncJobResult&response=json
   > 2023-01-05 17:52:11,772 DEBUG [c.c.a.ApiServer] (qtp1747352992-8705:ctx-d2189185 ctx-cbaabce4) (logid:b61b54a5) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d
   > "}' is allowed to perform API calls: 0.0.0.0/0,::/0
   > agent log: 2023-01-05 17:52:04,529 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-2:null) (logid:564609bc) Trying to connect to 169.254.236.18 2023-01-05 17:52:05,532 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py get_rule_logs_for_vms 2023-01-05 17:52:05,534 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Executing while with timeout : 1800000 2023-01-05 17:52:05,624 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Execution is successful. 2023-01-05 17:52:05,625 DEBUG [kvm.resource.LibvirtConnection] (UgentTask-5:null) (logid:) Looking for libvirtd connection at: qemu:///system 2023-01-05 17:52:05,641 DEBUG [cloud.agent.Agent] (UgentTask-5:null) (logid:) Sending ping: Seq 1-438: { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 11, [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupState
 s":{},"_hostVmStateReport":{"v-88-VM":{"state":" PowerOn","host":"cluster13"},"r-99-VM":{"state":"PowerOn","host":"cluster13"},"s-89-VM":{"state":"PowerOn","host":"cluster13"}},"_gatewayAccessible":"true","_vnetAccessible":"true","hostType":"Routing","hostId":"1","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:05,709 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null) (logid:05ee669b) Received response: Seq 1-438: { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":"1"," wait":"0","bypassHostMaintenance":"false"},"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:08,514 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:564609bc) Processing command: com.cloud.agent.api.check.CheckSshCommand 2023-01-05 17:52:08,514 DEBUG [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] (agentRequest-Handler-2:null) (logid:564609bc) Ping com
 mand port, 169.254.236.18:3922 2023-01-05 17:52:08,514 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-2:null) (logid:564609bc) Trying to connect to 169.254.236.18 2023-01-05 17:52:08,515 DEBUG [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] (agentRequest-Handler-2:null) (logid:564609bc) Ping command port succeeded for vm r-99-VM 2023-01-05 17:52:08,515 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:564609bc) Processing command: com.cloud.agent.api.GetDomRVersionCmd 2023-01-05 17:52:08,515 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh get_template_version.sh 169.254.236.18 2023-01-05 17:52:08,517 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing while with timeout : 1800000 2023-01-05 17:52:08,598 DEBUG [kvm.resource.LibvirtComputingResource] (age
 ntRequest-Handler-2:null) (logid:564609bc) Exit value is 255 2023-01-05 17:52:08,599 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) 2023-01-05 17:52:08,599 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing script in VR: get_template_version.sh 2023-01-05 17:52:08,613 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:564609bc) Seq 1-4745668107341661094: { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":"99","name":"r-99-VM","state":"Star ting","type":"DomainRouter","cpus":"1","minSpeed":"500","maxSpeed":"500","minRam":"(256.00 MB) 268435456","maxRam":"(256.00 MB) 268435456","arch":"x86_64","os":"Debian GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux 5","bootArgs":" template=domP name=r-99-VM authorized_key=c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCZ1FEZmcxeUV5b0VONWtOZEd4SVcwYlJUWHN1RFJSd2ZBTDBhN
 Hdpa2Y5QXhrdWZTSEJhMnhTSmJmWm5mcXpvN1FmNDBlempzcm1UTDE4bzZyT1NkQUE4cFN1NHpXcTh0Rzg1VWszSDA5bkkrdSs4eGtlNlFvckhaK3JUZVVjTFh5WU1wSGd1UEJTeUxZSGhMelllMlJSRXRVRU5lWW9DbndCT0hlbmVsZTNWTnZrM3gwR2dpUTdpTE41RXpqQlpVSTgzMnYxcDNXcHVNQnF3ZEhlcVdxdUNmMEdXNGlUc0gzejZDZXhXakhXc21Fc3VEbTBNU3ZrWEszR2V2RWtUZytiL0hrRXBpV2NOMnBQb3ErcGU5N2dLV3I5UW40U2x6ZnUyZHZpZXdQdWJzSFBiMGFzeHdWRHB3eVp0M1cwMktBaHhSVEtZZkxnVWFJSGZnZWFFVzRLUDZpQ3V5bVdoM3lzYVBvUVE3S1JJd3BjR003STVFQmEvV1hybDlkcU14K2pLV3A4MjJOQ0ZlUmNGazFPbUtBNTY2ci9DLzdvUTZLc1VVWTEzQVByenNMN09NaUdMdXYzaFlhNlBkN0FHdnd5SEcvVE42dml3QlMrVXhJMDFHaktxL0V2UTZPaWhQanBTVXpWU3ZZdnRkRCt5TDFDUXVSQ1RROWtnR2lRTFl6RXM9IGNsb3VkQGNsdXN0ZXIxMwo= eth0ip=192.168.0.78 eth0mask=255.255.255.0 gateway=192.168.0.200 domain=cs1cloud.internal cidrsize=24 dhcprange=192.168.0.1 eth1ip=169.254.236.18 eth1mask=255.255.0.0 type=dhcpsrvr disable_rp_filter=true dns1=8.8.8.8 exposedns=true baremetalnotificationsecuritykey=dQVh_AIeRj6Bnqmy4Yk5h2PqeKd6STcTareyLEPQImE9J56kGuDrrNJfEHfi_0iPny1j
 LzntlTxsTHLpTYOzfg baremetalnotificationapikey=yQ6xRwhUH7pN7LmCTn5-Jp9saoc3t6vdrre3VSqb4basX1ccm9EN-iSC17g7HuBQiN2Ow03jYEvLSy7spt1h1A host=192.168.0.13 port=8080","enableHA":"true","limitCpuUse":"false","enableDynamicallyScaleVm":"false","vncPassword":"Dmz6dCAEg-7xINbmS8HlIg","vncAddr":"192.168.0.13","params":{},"uuid":"4fcea489-51f8-4bd9-baef-65dc22828a94","enterHardwareSetup":"false","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"68bbe57a-a958-430f-8a97-f3a4cc2d4b4e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9c23d00b-a619-4842-a7d1-045727e46f0e","name":"cluster13-local-9c23d00b","id":"1","poolType":"Filesystem","host":"192.168.0.13","path":"/var/lib/libvirt/images","port":"0","url":"Filesystem://192.168.0.13/var/lib/libvirt/images/?ROLE=Primary&STOREUUID=9c23d00b-a619-4842-a7d1-045727e46f0e","isManaged":"false"}},"name":"ROOT-99","size":"(3.91 GB) 4194304000","path":"68bbe57a-a958-430f-8a97-f3a4cc2
 d4b4e","volumeId":"80","vmName":"r-99-VM","accountId":"1","format":"QCOW2","provisioningType":"THIN","poolId":"1","id":"80","deviceId":"0","bytesReadRate":"(0 bytes) 0","bytesWriteRate":"(0 bytes) 0","iopsReadRate":"(0 bytes) 0","iopsWriteRate":"(0 bytes) 0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"diskSeq":"0","path":"68bbe57a-a958-430f-8a97-f3a4cc2d4b4e","type":"ROOT","_details":{"storageHost":"192.168.0.13","managed":"false","storagePort":"0","storage.pool.disk.wait":"60","volumeSize":"(3.91 GB) 4194304000"}}],"nics":[{"deviceId":"0","networkRateMbps":"200","defaultNic":"true","pxeDisable":"true","nicUuid":"599503a1-23c4-436e-8bc9-cbf0bc3bda53","details":{"MacLearning":"false","MacAddressChanges":"true","PromiscuousMode":"false","ForgedTransmits":"true"},"dpdkEnabled":"false","uuid":"808f347a-c4ec-45fb-bd7d-73b774fecc96","ip":"192.168.0.78","netmask":"255.255.255.0","gateway":"192.168.0.200","mac":"1e:00:34:00:00:14","dns1":"8.8.8.8","broadcastType"
 :"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":"false"},{"deviceId":"1","networkRateMbps":"-1","defaultNic":"false","pxeDisable":"true","nicUuid":"88248332-e7de-47f4-8b2c-c339fb0debe6","details":{"MacLearning":"false","MacAddressChanges":"true","PromiscuousMode":"false","ForgedTransmits":"true"},"dpdkEnabled":"false","uuid":"c529c6ad-fc5f-4026-b91a-51514ee1e0f4","ip":"169.254.236.18","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:ec:12","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":"false"}],"vcpuMaxLimit":"1","configDriveLocation":"SECONDARY","guestOsDetails":{},"extraConfig":{}},"result":"true","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"result":"false","details":"GetDomRVersionCmd failed","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.
 api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:08,724 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:564609bc) Request:Seq 1-4745668107341661096: { Cmd , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeClean up":"false","controlIp":"169.254.144.182","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"r-99-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance"
 :"false"}}] }2023-01-05 17:52:08,725 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:564609bc) Processing command: com.cloud.agent.api.StopCommand 2023-01-05 17:52:08,725 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system 2023-01-05 17:52:08,746 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname r-99-VM --vif vnet52 2023-01-05 17:52:08,747 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 1800000 2023-01-05 17:52:08,841 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful. 2023-01-05 17:52:08,842 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Cleani
 ng the metadata of vm snapshots of vm r-99-VM 2023-01-05 17:52:08,843 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Try to stop the vm at first 2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) successfully shut down vm r-99-VM 2023-01-05 17:52:10,951 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:564609bc) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied. 2023-01-05 17:52:10,951 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:564609bc) Groovy scripting engine is not initialized. Data transformation skipped. 2023-01-05 17:52:10,951 DEBUG [kvm.storage.KVMStoragePoolManager] (agentRequest-Handler-5:null) (logid:564609bc) Disconnect physical disk by path: /var/lib/libvirt/images/68bbe57a-a958-430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,95
 1 DEBUG [kvm.storage.StorPoolStorageAdaptor] (agentRequest-Handler-5:null) (logid:564609bc) disconnectPhysicalDiskByPath: localPath=/var/lib/libvirt/images/68bbe57a-a958-430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Unable to clean up disk with null path (perhaps empty cdrom drive):
   > 
   > 2023-01-05 17:52:10,951 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloudbr0 2023-01-05 17:52:10,953 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000 2023-01-05 17:52:10,957 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful. 2023-01-05 17:52:10,958 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloudbr0/brif | tr ' ' ' ' 2023-01-05 17:52:10,960 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000 2023-01-05 17:52:10,964 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful. 2023-01-05 17:52:10,965 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloud0 202
 3-01-05 17:52:10,966 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000 2023-01-05 17:52:10,969 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful. 2023-01-05 17:52:10,969 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloud0/brif | tr ' ' ' ' 2023-01-05 17:52:10,970 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000 2023-01-05 17:52:10,974 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful. 2023-01-05 17:52:10,978 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:564609bc) Seq 1-4745668107341661096: { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] }2023-01-05 17:52:10,998 
 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:564609bc) Request:Seq 1-4745668107341661097: { Cmd , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeClean up":"false","controlIp":"169.254.144.182","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"r-99-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:10,998 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:564609bc) Processing command: com.cloud.agent.api.StopCommand 2023-01-05 17:52:10,998 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system 2023-01-05 17:52:11,011 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Can not find KVM connection for Instance: r-99-VM, continuing. 2023-01-05 17:52:11,011 DEBUG [kvm.resource.
 LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Looking for libvirtd connection at: lxc:/// 2023-01-05 17:52:11,011 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) No existing libvirtd connection found. Opening a new one 2023-01-05 17:52:11,013 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Can not find LXC connection for Instance: r-99-VM, continuing. 2023-01-05 17:52:11,013 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Can not find a connection for Instance r-99-VM. Assuming the default connection. 2023-01-05 17:52:11,013 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system 2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain
  with matching name 'r-99-VM' 2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'r-99-VM' 2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'r-99-VM' 2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname r-99-VM 2023-01-05 17:52:11,027 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Executing while with timeout : 1800000 2023-01-05 17:52:11,124 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) 
 Execution is successful. 2023-01-05 17:52:11,125 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get vm :Domain not found: no domain with matching name 'r-99-VM' 2023-01-05 17:52:11,125 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Try to stop the vm at first 2023-01-05 17:52:11,126 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) VM r-99-VM doesn't exist, no need to stop it 2023-01-05 17:52:11,126 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-1:null) (logid:564609bc) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied. 2023-01-05 17:52:11,126 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-1:null) (logid:564609bc) Groovy scripting engine is not initialized. Data transformation skipped. 2023-01-05 17:52:11,126 DEBUG [cloud.agent.Agent
 ] (agentRequest-Handler-1:null) (logid:564609bc) Seq 1-4745668107341661097: { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] }2023-01-05 17:52:11,239 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Request:Seq 1-4745668107341661098: { Cmd , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeClean up":"false","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"i-2-101-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:11,239 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Processing command: com.cloud.agent.api.StopCommand 2023-01-05 17:52:11,239 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Looking for libvirtd connection a
 t: qemu:///system 2023-01-05 17:52:11,252 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Can not find KVM connection for Instance: i-2-101-VM, continuing. 2023-01-05 17:52:11,252 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Looking for libvirtd connection at: lxc:/// 2023-01-05 17:52:11,252 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) No existing libvirtd connection found. Opening a new one 2023-01-05 17:52:11,253 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Can not find LXC connection for Instance: i-2-101-VM, continuing. 2023-01-05 17:52:11,253 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Can not find a connection for Instance i-2-101-VM. Assuming the default connection. 2023-01-05 17:52:11,253 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Look
 ing for libvirtd connection at: qemu:///system 2023-01-05 17:52:11,265 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'i-2-101-VM' 2023-01-05 17:52:11,266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'i-2-101-VM' 2023-01-05 17:52:11,266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'i-2-101-VM' 2023-01-05 17:52:11,266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname i-2-101-VM 2023-01-05 17:52:
 11,267 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Executing while with timeout : 1800000 2023-01-05 17:52:11,448 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Execution is successful. 2023-01-05 17:52:11,449 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) ebtables: No chain/target/match by that name ebtables: No chain/target/match by that name ebtables: No chain/target/match by that name ebtables: No chain/target/match by that name ebtables: No chain/target/match by that name ebtables: No chain/target/match by that name iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. ipset v7.1: The set with the given name does not exis
 t
   > 
   > 2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get vm :Domain not found: no domain with matching name 'i-2-101-VM' 2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Try to stop the vm at first 2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) VM i-2-101-VM doesn't exist, no need to stop it 2023-01-05 17:52:11,450 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4:null) (logid:564609bc) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied. 2023-01-05 17:52:11,451 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4:null) (logid:564609bc) Groovy scripting engine is not initialized. Data transformation skipped. 2023-01-05 17:52:11,451 DEBUG [cloud.agent.Agent] (agentRequest
 -Handler-4:null) (logid:564609bc) Seq 1-4745668107341661098: { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] }2023-01-05 17:52:33,939 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:d8ff1a24) Processing command: com.cloud.agent.api.GetStorageStatsCommand 2023-01-05 17:52:33,940 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:d8ff1a24) Trying to fetch storage pool 9c23d00b-a619-4842-a7d1-045727e46f0e from libvirt
   
   @fangyi1008 
   there was an issue while start the VR.
   
   ```
   2023-01-05 17:52:08,515 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh get_template_version.sh 169.254.236.18
   2023-01-05 17:52:08,517 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing while with timeout : 1800000
   2023-01-05 17:52:08,598 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Exit value is 255
   ```
   
   can you check if the VR is created from correct template ?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] sbrueseke commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
sbrueseke commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371978619

   Does your service offering use local storage?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] weizhouapache commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1373554925

   closing in favor of #7064 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] boring-cyborg[bot] commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371742898

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] weizhouapache closed issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
weizhouapache closed issue #7053: cant not instance
URL: https://github.com/apache/cloudstack/issues/7053


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] DaanHoogland commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371939672

   there is no suitable destination host to deploy your instance on. please check if there is enough free capacity to deploy an instance of the size you specified.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1372006451

   > can you share more logs, please ?
   management log:
   
   2023-01-05 17:52:11,528 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 101, job origin: 446
   2023-01-05 17:52:11,528 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) Unable to complete AsyncJobVO: {id:447, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: 
   rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAZXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 2618337734
 22727, completeMsid: null, lastUpdated: null, lastPolled: null, created: Thu Jan 05 17:51:56 CST 2023, removed: null}, job origin:446com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1140)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source)
   	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
   	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
   	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   2023-01-05 17:52:11,541 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) Remove job-447 from job monitoring
   2023-01-05 17:52:11,554 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Destroying vm VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"} as it failed to create 
   on Host with Id:12023-01-05 17:52:11,571 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"} state transited from [Stopped] to [
   Error] with event [OperationFailedToError]. VM's original host: null, new host: null, host before state transition: null2023-01-05 17:52:11,586 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = volume count for Account = 2 Operation = decreasing Amount = 1
   2023-01-05 17:52:11,595 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = primary_storage count for Account = 2 Operation = decreasing Amount = (20.00 GB) 21474836480
   2023-01-05 17:52:11,614 WARN  [c.c.a.AlertManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) alertType=[8] dataCenterId=[1] podId=[1] clusterId=[null] message=[Failed to deploy Vm with Id: 101, on Host with Id: 1].
   2023-01-05 17:52:11,621 WARN  [c.c.a.AlertManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Failed to deploy Vm with Id: 101, on Host w
   ith Id: 1] and content [Failed to deploy Vm with Id: 101, on Host with Id: 1].2023-01-05 17:52:11,622 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = user_vm count for Account = 2 Operation = decreasing Amount = 1
   2023-01-05 17:52:11,630 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = cpu count for Account = 2 Operation = decreasing Amount = 1
   2023-01-05 17:52:11,637 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) Updating resource Type = memory count for Account = 2 Operation = decreasing Amount = 512
   2023-01-05 17:52:11,653 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-117:ctx-f896976b job-446) (logid:564609bc) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin
   com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM [60093c58-bba6-470e-a958-0d30e5507628] due to [Unable to create a deployment for VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}].
   	at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:803)
   	at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
   	at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:214)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5311)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5161)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4788)
   	at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4777)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   	at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
   	at com.sun.proxy.$Proxy180.startVirtualMachine(Unknown Source)
   	at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:714)
   	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163)
   	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:106)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1140)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source)
   	... 17 more
   2023-01-05 17:52:11,664 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-117:ctx-f896976b job-446) (logid:564609bc) Remove job-446 from job monitoring
   2023-01-05 17:52:11,768 DEBUG [c.c.a.ApiServlet] (qtp1747352992-8705:ctx-d2189185) (logid:b61b54a5) ===START===  192.168.0.102 -- GET  jobId=564609bc-a5e0-43eb-a1a9-45f5513d3ee0&command=queryAsyncJobResult&response=json
   2023-01-05 17:52:11,772 DEBUG [c.c.a.ApiServer] (qtp1747352992-8705:ctx-d2189185 ctx-cbaabce4) (logid:b61b54a5) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d
   "}' is allowed to perform API calls: 0.0.0.0/0,::/0
   ------------------------------------------------------------------------------------------------------------------------------------------
   agent log:
   2023-01-05 17:52:04,529 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-2:null) (logid:564609bc) Trying to connect to 169.254.236.18
   2023-01-05 17:52:05,532 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py get_rule_logs_for_vms 
   2023-01-05 17:52:05,534 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Executing while with timeout : 1800000
   2023-01-05 17:52:05,624 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Execution is successful.
   2023-01-05 17:52:05,625 DEBUG [kvm.resource.LibvirtConnection] (UgentTask-5:null) (logid:) Looking for libvirtd connection at: qemu:///system
   2023-01-05 17:52:05,641 DEBUG [cloud.agent.Agent] (UgentTask-5:null) (logid:) Sending ping: Seq 1-438:  { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 11, [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{"v-88-VM":{"state":"
   PowerOn","host":"cluster13"},"r-99-VM":{"state":"PowerOn","host":"cluster13"},"s-89-VM":{"state":"PowerOn","host":"cluster13"}},"_gatewayAccessible":"true","_vnetAccessible":"true","hostType":"Routing","hostId":"1","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:05,709 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null) (logid:05ee669b) Received response: Seq 1-438:  { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":"1","
   wait":"0","bypassHostMaintenance":"false"},"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:08,514 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:564609bc) Processing command: com.cloud.agent.api.check.CheckSshCommand
   2023-01-05 17:52:08,514 DEBUG [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] (agentRequest-Handler-2:null) (logid:564609bc) Ping command port, 169.254.236.18:3922
   2023-01-05 17:52:08,514 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-2:null) (logid:564609bc) Trying to connect to 169.254.236.18
   2023-01-05 17:52:08,515 DEBUG [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] (agentRequest-Handler-2:null) (logid:564609bc) Ping command port succeeded for vm r-99-VM
   2023-01-05 17:52:08,515 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:564609bc) Processing command: com.cloud.agent.api.GetDomRVersionCmd
   2023-01-05 17:52:08,515 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh get_template_version.sh 169.254.236.18 
   2023-01-05 17:52:08,517 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing while with timeout : 1800000
   2023-01-05 17:52:08,598 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Exit value is 255
   2023-01-05 17:52:08,599 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) 
   2023-01-05 17:52:08,599 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing script in VR: get_template_version.sh
   2023-01-05 17:52:08,613 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:564609bc) Seq 1-4745668107341661094:  { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":"99","name":"r-99-VM","state":"Star
   ting","type":"DomainRouter","cpus":"1","minSpeed":"500","maxSpeed":"500","minRam":"(256.00 MB) 268435456","maxRam":"(256.00 MB) 268435456","arch":"x86_64","os":"Debian GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux 5","bootArgs":" template=domP name=r-99-VM authorized_key=c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCZ1FEZmcxeUV5b0VONWtOZEd4SVcwYlJUWHN1RFJSd2ZBTDBhNHdpa2Y5QXhrdWZTSEJhMnhTSmJmWm5mcXpvN1FmNDBlempzcm1UTDE4bzZyT1NkQUE4cFN1NHpXcTh0Rzg1VWszSDA5bkkrdSs4eGtlNlFvckhaK3JUZVVjTFh5WU1wSGd1UEJTeUxZSGhMelllMlJSRXRVRU5lWW9DbndCT0hlbmVsZTNWTnZrM3gwR2dpUTdpTE41RXpqQlpVSTgzMnYxcDNXcHVNQnF3ZEhlcVdxdUNmMEdXNGlUc0gzejZDZXhXakhXc21Fc3VEbTBNU3ZrWEszR2V2RWtUZytiL0hrRXBpV2NOMnBQb3ErcGU5N2dLV3I5UW40U2x6ZnUyZHZpZXdQdWJzSFBiMGFzeHdWRHB3eVp0M1cwMktBaHhSVEtZZkxnVWFJSGZnZWFFVzRLUDZpQ3V5bVdoM3lzYVBvUVE3S1JJd3BjR003STVFQmEvV1hybDlkcU14K2pLV3A4MjJOQ0ZlUmNGazFPbUtBNTY2ci9DLzdvUTZLc1VVWTEzQVByenNMN09NaUdMdXYzaFlhNlBkN0FHdnd5SEcvVE42dml3QlMrVXhJMDFHaktxL0V2UTZPaWhQanBTVXpWU3ZZdnRkRCt5
 TDFDUXVSQ1RROWtnR2lRTFl6RXM9IGNsb3VkQGNsdXN0ZXIxMwo= eth0ip=192.168.0.78 eth0mask=255.255.255.0 gateway=192.168.0.200 domain=cs1cloud.internal cidrsize=24 dhcprange=192.168.0.1 eth1ip=169.254.236.18 eth1mask=255.255.0.0 type=dhcpsrvr disable_rp_filter=true dns1=8.8.8.8 exposedns=true baremetalnotificationsecuritykey=dQVh_AIeRj6Bnqmy4Yk5h2PqeKd6STcTareyLEPQImE9J56kGuDrrNJfEHfi_0iPny1jLzntlTxsTHLpTYOzfg baremetalnotificationapikey=yQ6xRwhUH7pN7LmCTn5-Jp9saoc3t6vdrre3VSqb4basX1ccm9EN-iSC17g7HuBQiN2Ow03jYEvLSy7spt1h1A host=192.168.0.13 port=8080","enableHA":"true","limitCpuUse":"false","enableDynamicallyScaleVm":"false","vncPassword":"Dmz6dCAEg-7xINbmS8HlIg","vncAddr":"192.168.0.13","params":{},"uuid":"4fcea489-51f8-4bd9-baef-65dc22828a94","enterHardwareSetup":"false","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"68bbe57a-a958-430f-8a97-f3a4cc2d4b4e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9c23d00b-a6
 19-4842-a7d1-045727e46f0e","name":"cluster13-local-9c23d00b","id":"1","poolType":"Filesystem","host":"192.168.0.13","path":"/var/lib/libvirt/images","port":"0","url":"Filesystem://192.168.0.13/var/lib/libvirt/images/?ROLE=Primary&STOREUUID=9c23d00b-a619-4842-a7d1-045727e46f0e","isManaged":"false"}},"name":"ROOT-99","size":"(3.91 GB) 4194304000","path":"68bbe57a-a958-430f-8a97-f3a4cc2d4b4e","volumeId":"80","vmName":"r-99-VM","accountId":"1","format":"QCOW2","provisioningType":"THIN","poolId":"1","id":"80","deviceId":"0","bytesReadRate":"(0 bytes) 0","bytesWriteRate":"(0 bytes) 0","iopsReadRate":"(0 bytes) 0","iopsWriteRate":"(0 bytes) 0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"diskSeq":"0","path":"68bbe57a-a958-430f-8a97-f3a4cc2d4b4e","type":"ROOT","_details":{"storageHost":"192.168.0.13","managed":"false","storagePort":"0","storage.pool.disk.wait":"60","volumeSize":"(3.91 GB) 4194304000"}}],"nics":[{"deviceId":"0","networkRateMbps":"200","defaultNic":
 "true","pxeDisable":"true","nicUuid":"599503a1-23c4-436e-8bc9-cbf0bc3bda53","details":{"MacLearning":"false","MacAddressChanges":"true","PromiscuousMode":"false","ForgedTransmits":"true"},"dpdkEnabled":"false","uuid":"808f347a-c4ec-45fb-bd7d-73b774fecc96","ip":"192.168.0.78","netmask":"255.255.255.0","gateway":"192.168.0.200","mac":"1e:00:34:00:00:14","dns1":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":"false"},{"deviceId":"1","networkRateMbps":"-1","defaultNic":"false","pxeDisable":"true","nicUuid":"88248332-e7de-47f4-8b2c-c339fb0debe6","details":{"MacLearning":"false","MacAddressChanges":"true","PromiscuousMode":"false","ForgedTransmits":"true"},"dpdkEnabled":"false","uuid":"c529c6ad-fc5f-4026-b91a-51514ee1e0f4","ip":"169.254.236.18","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:ec:12","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":"false"}],"vcpuMaxLimit":"1","configDriveLoc
 ation":"SECONDARY","guestOsDetails":{},"extraConfig":{}},"result":"true","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"result":"false","details":"GetDomRVersionCmd failed","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.Answer":{"result":"false","details":"Stopped by previous failure","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:08,724 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:564609bc
 ) Request:Seq 1-4745668107341661096:  { Cmd , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeClean
   up":"false","controlIp":"169.254.144.182","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"r-99-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:08,725 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:564609bc) Processing command: com.cloud.agent.api.StopCommand
   2023-01-05 17:52:08,725 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system
   2023-01-05 17:52:08,746 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname r-99-VM --vif vnet52 
   2023-01-05 17:52:08,747 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 1800000
   2023-01-05 17:52:08,841 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:08,842 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Cleaning the metadata of vm snapshots of vm r-99-VM
   2023-01-05 17:52:08,843 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Try to stop the vm at first
   2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) successfully shut down vm r-99-VM
   2023-01-05 17:52:10,951 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:564609bc) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
   2023-01-05 17:52:10,951 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:564609bc) Groovy scripting engine is not initialized. Data transformation skipped.
   2023-01-05 17:52:10,951 DEBUG [kvm.storage.KVMStoragePoolManager] (agentRequest-Handler-5:null) (logid:564609bc) Disconnect physical disk by path: /var/lib/libvirt/images/68bbe57a-a958-430f-8a97-f3a4cc2d4b4e
   2023-01-05 17:52:10,951 DEBUG [kvm.storage.StorPoolStorageAdaptor] (agentRequest-Handler-5:null) (logid:564609bc) disconnectPhysicalDiskByPath: localPath=/var/lib/libvirt/images/68bbe57a-a958-430f-8a97-f3a4cc2d4b4e
   2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) Unable to clean up disk with null path (perhaps empty cdrom drive):<disk  device='cdrom' type='file'>
   <driver name='qemu' type='raw' />
   <source file=''/>
   <target dev='hdb' bus='ide'/>
   </disk>
   
   2023-01-05 17:52:10,951 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloudbr0 
   2023-01-05 17:52:10,953 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000
   2023-01-05 17:52:10,957 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:10,958 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloudbr0/brif | tr '
   ' ' ' 
   2023-01-05 17:52:10,960 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000
   2023-01-05 17:52:10,964 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:10,965 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloud0 
   2023-01-05 17:52:10,966 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000
   2023-01-05 17:52:10,969 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:10,969 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing: /bin/bash -c ls /sys/class/net/cloud0/brif | tr '
   ' ' ' 
   2023-01-05 17:52:10,970 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Executing while with timeout : 3600000
   2023-01-05 17:52:10,974 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:10,978 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:564609bc) Seq 1-4745668107341661096:  { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan
   ce":"false"}}] }2023-01-05 17:52:10,998 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:564609bc) Request:Seq 1-4745668107341661097:  { Cmd , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeClean
   up":"false","controlIp":"169.254.144.182","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"r-99-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:10,998 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:564609bc) Processing command: com.cloud.agent.api.StopCommand
   2023-01-05 17:52:10,998 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system
   2023-01-05 17:52:11,011 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Can not find KVM connection for Instance: r-99-VM, continuing.
   2023-01-05 17:52:11,011 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Looking for libvirtd connection at: lxc:///
   2023-01-05 17:52:11,011 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) No existing libvirtd connection found. Opening a new one
   2023-01-05 17:52:11,013 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Can not find LXC connection for Instance: r-99-VM, continuing.
   2023-01-05 17:52:11,013 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Can not find a connection for Instance r-99-VM. Assuming the default connection.
   2023-01-05 17:52:11,013 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system
   2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'r-99-VM'
   2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'r-99-VM'
   2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'r-99-VM'
   2023-01-05 17:52:11,026 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname r-99-VM 
   2023-01-05 17:52:11,027 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Executing while with timeout : 1800000
   2023-01-05 17:52:11,124 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:11,125 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Failed to get vm :Domain not found: no domain with matching name 'r-99-VM'
   2023-01-05 17:52:11,125 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) Try to stop the vm at first
   2023-01-05 17:52:11,126 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) VM r-99-VM doesn't exist, no need to stop it
   2023-01-05 17:52:11,126 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-1:null) (logid:564609bc) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
   2023-01-05 17:52:11,126 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-1:null) (logid:564609bc) Groovy scripting engine is not initialized. Data transformation skipped.
   2023-01-05 17:52:11,126 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:564609bc) Seq 1-4745668107341661097:  { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan
   ce":"false"}}] }2023-01-05 17:52:11,239 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Request:Seq 1-4745668107341661098:  { Cmd , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeClean
   up":"false","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"i-2-101-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 17:52:11,239 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Processing command: com.cloud.agent.api.StopCommand
   2023-01-05 17:52:11,239 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system
   2023-01-05 17:52:11,252 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Can not find KVM connection for Instance: i-2-101-VM, continuing.
   2023-01-05 17:52:11,252 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Looking for libvirtd connection at: lxc:///
   2023-01-05 17:52:11,252 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) No existing libvirtd connection found. Opening a new one
   2023-01-05 17:52:11,253 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Can not find LXC connection for Instance: i-2-101-VM, continuing.
   2023-01-05 17:52:11,253 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Can not find a connection for Instance i-2-101-VM. Assuming the default connection.
   2023-01-05 17:52:11,253 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) Looking for libvirtd connection at: qemu:///system
   2023-01-05 17:52:11,265 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'i-2-101-VM'
   2023-01-05 17:52:11,266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'i-2-101-VM'
   2023-01-05 17:52:11,266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 'i-2-101-VM'
   2023-01-05 17:52:11,266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname i-2-101-VM 
   2023-01-05 17:52:11,267 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Executing while with timeout : 1800000
   2023-01-05 17:52:11,448 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Execution is successful.
   2023-01-05 17:52:11,449 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) ebtables: No chain/target/match by that name
   ebtables: No chain/target/match by that name
   ebtables: No chain/target/match by that name
   ebtables: No chain/target/match by that name
   ebtables: No chain/target/match by that name
   ebtables: No chain/target/match by that name
   iptables: No chain/target/match by that name.
   iptables: No chain/target/match by that name.
   iptables: No chain/target/match by that name.
   iptables: No chain/target/match by that name.
   iptables: No chain/target/match by that name.
   iptables: No chain/target/match by that name.
   ipset v7.1: The set with the given name does not exist
   
   2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Failed to get vm :Domain not found: no domain with matching name 'i-2-101-VM'
   2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) Try to stop the vm at first
   2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) VM i-2-101-VM doesn't exist, no need to stop it
   2023-01-05 17:52:11,450 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4:null) (logid:564609bc) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
   2023-01-05 17:52:11,451 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4:null) (logid:564609bc) Groovy scripting engine is not initialized. Data transformation skipped.
   2023-01-05 17:52:11,451 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Seq 1-4745668107341661098:  { Ans: , MgmtId: 261833773422727, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan
   ce":"false"}}] }2023-01-05 17:52:33,939 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:d8ff1a24) Processing command: com.cloud.agent.api.GetStorageStatsCommand
   2023-01-05 17:52:33,940 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:d8ff1a24) Trying to fetch storage pool 9c23d00b-a619-4842-a7d1-045727e46f0e from libvirt


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1372040099

   > > > 你能分享更多日志吗?
   > > > 管理日志:
   > > 
   > > 
   > > ## 2023-01-05 17:52:11,528 DEBUG [ccvVmWorkJobDispatcher] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) 完成运行 VM 工作作业:com.cloud VM 101 的 .vm.VmWorkStart,作业来源:446
   > > 2023-01-05 17:52:11,528 错误 [ccvVmWorkJobDispatcher] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) 无法完成 AsyncJobVO: {id:447, userId : 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
   > > BRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,cmdVersion:0,状态:IN_PROGRESS,processStatus:0,resultCode:0,结果:null,initMsid:261833773422727,completeMsid:null,lastUpdated:null,lastPolled:null,创建时间:Thu Jan 05 17:51:5263 CST 5263 ,已删除:null},作业来源:446com.cloud.exception.InsufficientServerCapacityException:无法为 VM 实例创建部署 {id:“101”,名称:“i-2-101-VM”,uuid:“60093c58-bba6 -470e-a958-0d30e5507628", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1 InsufficientServerCapacityException: 无法为 VM 实例创建部署 {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}Scope =接口com.cloud.dc.DataCenter;id=1 InsufficientServerCapacityException: 无法为 VM 实例创建部署 {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"}Scope =接口com.cloud.dc.DataCenter;id=1
   > > 在 com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1140)
   > > 在 com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   > > 在 jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source)
   > > 在 java .base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   > > 在 java.base/java.lang.reflect.Method.invoke(Method.java:566)
   > > 在 com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob (VmWorkJobHandlerProxy.java:107)
   > > 在 com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
   > > 在 com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
   > > 在 org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   > > 在 org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   > > 在 org.apache .cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   > > 在 org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   > > 在 org.apache.cloudstack。 managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   > > 在 org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   > > 在 org.apache.cloudstack.framework.jobs.impl。 AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   > > 在 java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   > > 在 java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   > > 在 java.base/java .util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   > > 在 java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   > > 在 java.base/java.lang.Thread.run (Thread.java:829) 
   > > 2023-01-05 17:52:11,541 INFO [oacfjiAsyncJobMonitor] (Work-Job-Executor-98:ctx-00ddfb01 job-446/job-447) (logid:564609bc) 删除作业 447从作业监控
   > > 2023-01-05 17:52:11,554 DEBUG [ccvUserVmManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) 销毁 vm VM 实例 {id: "101", name : "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"} 因为它无法
   > > 在 ID 为:12023-01-05 17:52 的主机上创建: 11,571 调试 [cccCapacityManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) VM 实例 {id: "101", name: "i-2-101-VM", uuid: "60093c58-bba6-470e-a958-0d30e5507628", type="User"} 状态从 [Stopped] 转变为 [
   > > 错误] 与事件 [OperationFailedToError]。VM的原始主机:null,新主机:null,状态转换前的主机:null2023-01-05 17:52:11,586 DEBUG [ccrResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) ( logid:564609bc) 更新资源类型 = 帐户的数量 = 2 操作 = 减少数量 = 1 
   > > 2023-01-05 17:52:11,595 DEBUG [ccrResourceLimitManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx -c292f039) (logid:564609bc) 更新资源类型 = 帐户的主要存储计数 = 2 操作 = 减少数量 = (20.00 GB) 21474836480
   > > 2023-01-05 17:52:11,614 警告 [ccaAlertManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) alertType=[8] dataCenterId=[1] podId= [1] clusterId=[null] message=[无法在 ID 为 1 的主机上部署 ID 为 101 的虚拟机]。
   > > 2023-01-05 17:52:11,621 WARN [ccaAlertManagerImpl] (API-Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) 全局设置中未设置收件人 'alert.email.addresses ',跳过发送主题为 [无法在主机 w 上部署 ID 为 101 的 Vm 的警报
   > > ith Id: 1] 和内容 [无法在 ID 为 1 的主机上部署 ID 为 101 的 Vm].2023-01-05 17:52:11,622 DEBUG [ccrResourceLimitManagerImpl] (API-Job-Executor-117:ctx- f896976b job-446 ctx-c292f039) (logid:564609bc) 更新资源类型 = 帐户的 user_vm 计数 = 2 操作 = 减少数量 = 1 
   > > 2023-01-05 17:52:11,630 DEBUG [ccrResourceLimitManagerImpl] (API-Job-Executor- 117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) 更新资源类型 = 帐户的 cpu 计数 = 2 操作 = 减少数量 = 1 
   > > 2023-01-05 17:52:11,637 DEBUG [ccrResourceLimitManagerImpl] (API- Job-Executor-117:ctx-f896976b job-446 ctx-c292f039) (logid:564609bc) 更新资源类型 = 帐户的内存计数 = 2 操作 = 减少数量 = 512
   > > 2023-01-05 17:52:11,653 错误 [ccaApiAsyncJobDispatcher] (API-Job-Executor-117:ctx-f896976b job-446) (logid:564609bc) 执行 org.apache.cloudstack.api.command.admin 时出现意外异常.vm.DeployVMCmdByAdmin 
   > > com.cloud.utils.exception.CloudRuntimeException:无法启动 VM [60093c58-bba6-470e-a958-0d30e5507628] 由于 [无法为 VM 实例创建部署 {id:“101”,名称: “i-2-101-VM”,uuid:“60093c58-bba6-470e-a958-0d30e5507628”,类型 =“用户”}]。
   > > 在 com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:803)
   > > 在 org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
   > > 在 org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:214)
   > > 在 com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5311)
   > > 在 com.cloud.vm.UserVmManagerImpl .startVirtualMachine(UserVmManagerImpl.java:5161)
   > > 在 com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4788)
   > > 在 com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4777)
   > > 在 java.base/jdk。 internal.reflect.NativeMethodAccessorImpl.invoke0(本机方法)
   > > 在 java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   > > 在 java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java :43)
   > > 在 java.base/java.lang.reflect.Method.invoke(Method.java:566)
   > > 在 org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
   > > 在 org.springframework.aop.framework.ReflectiveMethodInvocation .invokeJoinpoint(ReflectiveMethodInvocation.java:198)
   > > 在 org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   > > 在 org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java :107)
   > > 在 org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   > > 在 com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
   > > 在 org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
   > > 在 org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
   > > 在 org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:186)
   > > 在 org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
   > > 在 com.sun.proxy.$Proxy180.startVirtualMachine(未知来源)
   > > 在 org.apache.cloudstack.api。命令.user.vm.DeployVMCmd.execute(DeployVMCmd.java:714)
   > > 在com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163)
   > > 在com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:106)
   > > 在 org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   > > 在 org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   > > 在 org.apache .cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   > > 在 org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   > > 在 org.apache.cloudstack。 managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   > > 在 org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   > > 在 org.apache.cloudstack.framework.jobs.impl。 AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   > > 在 java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   > > 在 java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   > > 在 java.base/java .util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   > > 在 java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   > > 在 java.base/java.lang.Thread.run (Thread.java:829)
   > > 原因:com.cloud.exception.InsufficientServerCapacityException:无法为 VM 实例创建部署 {id:“101”,名称:“i-2-101-VM”,uuid:“60093c58- bba6-470e-a958-0d30e5507628", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
   > > 在 com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1140)
   > > 在 com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   > > 在 jdk.internal.reflect.GeneratedMethodAccessor554.invoke(未知来源)
   > > ... 17 更多
   > > 2023-01-05 17:52:11,664 信息 [oacfjiAsyncJobMonitor] (API-Job-Executor-117:ctx-f896976b job-446) (logid:564609bc) 从作业监控中删除 job-446 
   > > 2023-01-05 17:52:11,768 DEBUG [ccaApiServlet] (qtp1747352992-8705:ctx-d2189185 ) (logid:b61b54a5) ===START=== 192.168.0.102 -- GET jobId=564609bc-a5e0-43eb-a1a9-45f5513d3ee0&command=queryAsyncJobResult&response=json
   > > 2023-01-05 17:52:11,772 DEBUG [ccaApiServer] (qtp1747352992-8705:ctx-d2189185 ctx-cbaabce4) (logid:b61b54a5) 来自哪个帐户的 CIDR [2217c707-8817-11ed-9cb1-dae1-0442admin] -- 账户{"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d 
   > > "}'允许执行API调用:0.0.0.0/0,:: /0
   > > 代理日志:2023-01-05 17:52:04,529 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-2:null) (logid:564609bc) 尝试连接到 169.254.236.18 2023-01-05 17:52 :05,532 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) 执行:/usr/share/cloudstack-common/scripts/vm/network/security_group.py get_rule_logs_for_vms 2023-01-05 17:52 :05,534 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) 超时执行:1800000 2023-01-05 17:52:05,624 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5: null) (logid:) 执行成功。2023-01-05 17:52:05,625 DEBUG [kvm.resource.LibvirtConnection] (UgentTask-5:null) (logid:) 在以下位置寻找 libvirtd 连接:qemu:///system 2023-01-05 17:52: 05,641 DEBUG [cloud.agent.Agent] (UgentTask-5:null) (logid: LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) 执行成功。2023-01-05 17:52:08,842 DEBUG [kvm.resource.LibvirtComputingResource] (ag
 entRequest-Handler-5:null) (logid:564609bc) 清理虚拟机 r-99-VM 虚拟机快照的元数据 2023-01-05 17:52:08,843 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) 首先尝试停止虚拟机 2023-01-05 17:52:10,951 DEBUG [kvm.resource. LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) 成功关闭 vm r-99-VM 2023-01-05 17:52:10,951 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5: null) (logid:564609bc) Groovy 脚本 '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' 不可用。不会应用转换。2023-01-05 17:52:10,951 警告 [kvm.resource。LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:564609bc) Groovy 脚本引擎未初始化。跳过数据转换。2023-01-05 17:52:10,951 DEBUG [kvm.storage.KVMStoragePoolManager] (agentRequest-Handler-5:null) (logid:564609bc) 通过路径断开物理磁盘:/var/lib/libvirt/images/68bbe57a-a958 -430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52
 :10,951 调试 [kvm.storage.StorPoolStorageAdaptor] (agentRequest-Handler-5:null) (logid:564609bc) disconnectPhysicalDiskByPath: localPath=/var/lib/libvirt/images /68bbe57a-a958-430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) 无法使用空路径清理磁盘(也许是空光驱):跳过数据转换。2023-01-05 17:52:10,951 DEBUG [kvm.storage.KVMStoragePoolManager] (agentRequest-Handler-5:null) (logid:564609bc) 通过路径断开物理磁盘:/var/lib/libvirt/images/68bbe57a-a958 -430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,951 调试 [kvm.storage.StorPoolStorageAdaptor] (agentRequest-Handler-5:null) (logid:564609bc) disconnectPhysicalDiskByPath: localPath=/var/lib/libvirt/images /68bbe57a-a958-430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) 无法使用空路径清理磁盘(也许是空光驱�
 �:跳过数据转换。2023-01-05 17:52:10,951 DEBUG [kvm.storage.KVMStoragePoolManager] (agentRequest-Handler-5:null) (logid:564609bc) 通过路径断开物理磁盘:/var/lib/libvirt/images/68bbe57a-a958 -430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,951 调试 [kvm.storage.StorPoolStorageAdaptor] (agentRequest-Handler-5:null) (logid:564609bc) disconnectPhysicalDiskByPath: localPath=/var/lib/libvirt/images /68bbe57a-a958-430f-8a97-f3a4cc2d4b4e 2023-01-05 17:52:10,951 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:null) (logid:564609bc) 无法使用空路径清理磁盘(也许是空光驱):
   > > 2023-01-05 17:52:10,951 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行:/bin/bash -c ls /sys/class/net/cloudbr0 2023 -01-05 17:52:10,953 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 超时执行:3600000 2023-01-05 17:52:10,957 DEBUG [utils .script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行成功。2023-01-05 17:52:10,958 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行:/bin/bash -c ls /sys/class/net/cloudbr0/简介 | tr ' ' ' ' 2023-01-05 17:52:10,960 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 超时执行:3600000 2023-01-05 17: 52:10,964 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行成功。2023-01-05 17:52:10,965 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行:/bin/bash -c ls /sys/class/net/cloud0 2023 -01-05 17:52:10,966 DEBUG [utils.script.Script] (
 agentRequest-Handler-5:null) (logid:564609bc) 超时执行:3600000 2023-01-05 17:52:10,969 DEBUG [utils .script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行成功。2023-01-05 17:52:10,969 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行:/bin/bash -c ls /sys/class/net/cloud0/简介 | tr ' ' ' ' 2023-01-05 17:52:10,970 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 超时执行:3600000 2023-01-05 17: 52:10,974 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:564609bc) 执行成功。011 信息 [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) 找不到现有的 libvirtd 连接。打开一个新的 2023-01-05 17:52:11,013 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) 找不到实例的 LXC 连接:r-99-VM,继续。2023-01-05 17:52:11,013 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564
 609bc) 找不到实例 r-99-VM 的连接。假设默认连接。2023-01-05 17:52:11,013 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:564609bc) 在以下位置寻找 libvirtd 连接:qemu:///system 2023-01-05 17 :52:11,026 DEBUG [kvm.resource.LibvirtComputingResource](agentRequest-Handler-1:null)(logid:564609bc)无法获取dom xml:org.libvirt.LibvirtException:找不到域:null) (logid:564609bc) Executing while with timeout : 1800000 2023-01-05 17:52:11,124 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) 执行成功。2023-01-05 17:52:11,125 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) 无法获取虚拟机:找不到域:没有匹配名称 'r-99 的域-VM' 2023-01-05 17:52:11,125 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) 首先尝试停止虚拟机 2023-01-05 17:52 :11,126 DEBUG [kvm.re
 source.LibvirtComputingResource] (agentRequest-Handler-1:null) (logid:564609bc) VM r-99-VM 不存在,无需停止它 2023-01-05 17:52:11,126警告 [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-1:null) (logid:564609bc) Groovy 脚本 ' /etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' 不可用。不会应用转换。2023-01-05 17:52:11,126 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-1:null) (logid:564609bc) Groovy 脚本引擎未初始化。跳过数据转换。2023-01-05 17:52:11,126 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:564609bc) Seq 1-4745668107341661097: { Ans: , MgmtId: 261833773422727, via: 1, Ver : v1, 标志: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] } 2023-01-05 17:52:11,239 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Request:Seq 1-4745668107341661098: { Cmd, MgmtId: 261833773422727, 通过: 1,版本:v1,标志:10001
 1,null) (logid:564609bc) 找不到实例的 KVM 连接:i-2-101-VM,继续。2023-01-05 17:52:11,252 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) 在以下位置寻找 libvirtd 连接:lxc:/// 2023-01-05 17: 52:11,252 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) 找不到现有的 libvirtd 连接。打开一个新的 2023-01-05 17:52:11,253 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) 找不到实例的 LXC 连接:i-2-101-虚拟机,继续。2023-01-05 17:52:11,253 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:564609bc) 找不到实例 i-2-101-VM 的连接。假设默认连接。2023-01-05 17:52:11,253 调试 [kvm.resource。266 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) 执行:/usr/share/cloudstack-common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --
 vmname i-2 -101-VM 2023-01-05 17:52:11,267 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) 超时执行:1800000 2023-01-05 17:52 :11,448 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) 执行成功。2023-01-05 17:52:11,449 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) ebtables:没有链/目标/匹配该名称 ebtables:没有链/目标/按那个名字匹配 ebtables:没有链/目标/匹配那个名字 ebtables:没有链/目标/匹配那个名字 ebtables:没有那个名字的链/目标/匹配 ebtables:没有那个名字的链/目标/匹配 iptables:没有那个名字的链/目标/匹配。iptables:没有那个名字的链/目标/匹配。iptables:没有那个名字的链/目标/匹配。iptables:没有那个名字的链/目标/匹配。iptables:没有那个名字的链/目标/匹配。iptables:没有那个名字的链/目�
 ��/匹配。ipset v7.1:具有给定名称的集合不存在
   > > 2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) 无法获取虚拟机:找不到域:没有匹配名称“i-2”的域-101-VM' 2023-01-05 17:52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) 首先尝试停止虚拟机 2023-01-05 17 :52:11,450 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) (logid:564609bc) VM i-2-101-VM 不存在,无需停止 2023-01-05 17 :52:11,450 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4:null) (logid:564609bc) Groovy 脚本 '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' 不是可用的。不会应用转换。2023-01-05 17:52:11,451 警告 [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4: null) (logid:564609bc) Groovy 脚本引擎未初始化。跳过数据转换。2023-01-05 17:52:11,451 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc)
  Seq 1-4745668107341661098: { Ans: , MgmtId: 261833773422727, via: 1, Ver : v1, 标志: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] } 2023-01-05 17:52:33,939 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:d8ff1a24) 处理命令:com.cloud.agent.api.GetStorageStatsCommand 2023-01-05 17 :52:33,940 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:d8ff1a24) 尝试从 libvirt 获取存储池 9c23d00b-a619-4842-a7d1-045727e46f0e 564609bc) Groovy 脚本引擎未初始化。跳过数据转换。2023-01-05 17:52:11,451 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Seq 1-4745668107341661098: { Ans: , MgmtId: 261833773422727, via: 1, Ver : v1, 标志: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] } 2023-01-05 17:52:33,939 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:d8ff1a24) 处理命令�
 �com.cloud.agent.api.GetStorageStatsCommand 2023-01-05 17 :52:33,940 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:d8ff1a24) 尝试从 libvirt 获取存储池 9c23d00b-a619-4842-a7d1-045727e46f0e 564609bc) Groovy 脚本引擎未初始化。跳过数据转换。2023-01-05 17:52:11,451 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) (logid:564609bc) Seq 1-4745668107341661098: { Ans: , MgmtId: 261833773422727, via: 1, Ver : v1, 标志: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenan ce":"false"}}] } 2023-01-05 17:52:33,939 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:d8ff1a24) 处理命令:com.cloud.agent.api.GetStorageStatsCommand 2023-01-05 17 :52:33,940 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:d8ff1a24) 尝试从 libvirt 获取存储池 9c23d00b-a619-4842-a7d1-045727e46f0e
   > 
   > @fangyi1008 启动 VR 时出现问题。
   > 
   > ```
   > 2023-01-05 17:52:08,515 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh get_template_version.sh 169.254.236.18
   > 2023-01-05 17:52:08,517 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Executing while with timeout : 1800000
   > 2023-01-05 17:52:08,598 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) (logid:564609bc) Exit value is 255
   > ```
   > 
   > 你能检查 VR 是否是从正确的模板创建的吗?
   
   ![image](https://user-images.githubusercontent.com/89246132/210758715-0d8713b4-d349-4ccd-84d6-9a7ef51cb8ac.png)
   <img width="283" alt="4abba8537f57dcba24e277670106efc" src="https://user-images.githubusercontent.com/89246132/210758743-01b9c6cf-b8d4-4472-b347-432f7a179f9e.png">
   `2023-01-05 18:26:51,015 ERROR [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) GetDomRVersionCmd failed
   2023-01-05 18:26:51,015 WARN  [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Command: com.cloud.agent.api.GetDomRVersionCommand failed while starting virtual router
   2023-01-05 18:26:51,015 INFO  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) The guru did not like the answers so stopping VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65dc2282
   8a94", type="DomainRouter"}2023-01-05 18:26:51,026 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Seq 1-4100808935697547316: Sending  { Cmd , MgmtId: 279278805475902, via: 1(cluster13), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCo
   mmand":{"isProxy":"false","checkBeforeCleanup":"false","controlIp":"169.254.37.251","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"r-99-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 18:26:51,188 DEBUG [c.c.a.ApiServlet] (qtp1747352992-37:ctx-558cfce4) (logid:aeb58cdb) ===START===  192.168.0.103 -- GET  jobId=1c5b362f-6ddc-4ee0-b0f7-6b5b2c8f9dc3&command=queryAsyncJobResult&response=json
   2023-01-05 18:26:51,193 DEBUG [c.c.a.ApiServer] (qtp1747352992-37:ctx-558cfce4 ctx-9d61d1fb) (logid:aeb58cdb) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d"}
   ' is allowed to perform API calls: 0.0.0.0/0,::/02023-01-05 18:26:51,202 DEBUG [c.c.a.ApiServlet] (qtp1747352992-37:ctx-558cfce4 ctx-9d61d1fb) (logid:aeb58cdb) ===END===  192.168.0.103 -- GET  jobId=1c5b362f-6ddc-4ee0-b0f7-6b5b2c8f9dc3&command=queryAsyncJobResult&response=json
   2023-01-05 18:26:52,210 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-52c7c052) (logid:04db3ddc) Detected management node left, id:7, nodeIP:192.168.0.13
   2023-01-05 18:26:52,210 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-52c7c052) (logid:04db3ddc) Trying to connect to 192.168.0.13
   2023-01-05 18:26:52,210 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-52c7c052) (logid:04db3ddc) Management node 7 is detected inactive by timestamp but is pingable
   2023-01-05 18:26:53,709 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0eb73d18) (logid:d790ec50) Detected management node left, id:7, nodeIP:192.168.0.13
   2023-01-05 18:26:53,709 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0eb73d18) (logid:d790ec50) Trying to connect to 192.168.0.13
   2023-01-05 18:26:53,709 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0eb73d18) (logid:d790ec50) Management node 7 is detected inactive by timestamp but is pingable
   2023-01-05 18:26:53,917 DEBUG [c.c.a.t.Request] (AgentManager-Handler-6:null) (logid:) Seq 1-4100808935697547316: Processing:  { Ans: , MgmtId: 279278805475902, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMainten
   ance":"false"}}] }2023-01-05 18:26:53,917 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Seq 1-4100808935697547316: Received:  { Ans: , MgmtId: 279278805475902, via: 1(cluster13), Ver: v1, Flags: 10, { StopAnswer } }
   2023-01-05 18:26:53,917 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Failed to start instance VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65dc22828a94", type="DomainRo
   uter"}com.cloud.utils.exception.ExecutionException: Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1299)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
   	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
   	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   2023-01-05 18:26:53,922 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Cleaning up resources for the vm VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65dc22828a94", type="
   DomainRouter"} in Starting state2023-01-05 18:26:53,934 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Seq 1-4100808935697547317: Sending  { Cmd , MgmtId: 279278805475902, via: 1(cluster13), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCo
   mmand":{"isProxy":"false","checkBeforeCleanup":"false","controlIp":"169.254.37.251","forceStop":"false","vlanToPersistenceMap":{"untagged":"true"},"volumesToDisconnect":[],"vmName":"r-99-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }2023-01-05 18:26:54,109 DEBUG [c.c.a.t.Request] (AgentManager-Handler-7:null) (logid:) Seq 1-4100808935697547317: Processing:  { Ans: , MgmtId: 279278805475902, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMainten
   ance":"false"}}] }2023-01-05 18:26:54,109 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Seq 1-4100808935697547317: Received:  { Ans: , MgmtId: 279278805475902, via: 1(cluster13), Ver: v1, Flags: 10, { StopAnswer } }
   2023-01-05 18:26:54,113 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Successfully updated user statistics as a part of domR VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f
   8-4bd9-baef-65dc22828a94", type="DomainRouter"} reboot/stop2023-01-05 18:26:54,119 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Service SecurityGroup is not supported in the network id=204
   2023-01-05 18:26:54,128 DEBUG [c.c.n.g.ControlNetworkGuru] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Released nic: NicProfile {"broadcastUri":null,"iPv4Address":null,"id":202,"reservationId":null,"vmId":99}
   2023-01-05 18:26:54,134 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Successfully released network resources for the VM VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65d
   c22828a94", type="DomainRouter"} in Starting state2023-01-05 18:26:54,138 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Successfully released storage resources for the VM VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65d
   c22828a94", type="DomainRouter"} in Starting state2023-01-05 18:26:54,138 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Successfully cleaned up resources for the VM VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65dc22828
   a94", type="DomainRouter"} in Starting state2023-01-05 18:26:54,150 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) VM instance {id: "99", name: "r-99-VM", uuid: "4fcea489-51f8-4bd9-baef-65dc22828a94", type="DomainRouter"} state transited from [St
   arting] to [Stopped] with event [OperationFailed]. VM's original host: null, new host: null, host before state transition: Host {"id": "1", "name": "cluster13", "uuid": "cf71da72-bd76-40b5-9f4a-98599b3359f3", "type"="Routing"}2023-01-05 18:26:54,154 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Hosts's actual total CPU: 256000000 and CPU after applying overprovisioning: 256000000
   2023-01-05 18:26:54,154 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Hosts's actual total RAM: (502.52 GB) 539572183040 and RAM after applying overprovisioning: (502.52 GB) 539572174848
   2023-01-05 18:26:54,154 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) release cpu from host: 1, old used: 1500,reserved: 0, actual total: 256000000, total with overprovisioning: 256000000; new used: 10
   00,reserved:0; movedfromreserved: false,moveToReserveredfalse2023-01-05 18:26:54,154 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) release mem from host: 1, old used: (1.75 GB) 1879048192,reserved: (0 bytes) 0, total: (502.52 GB) 539572174848; new used: (1.50 GB
   ) 1610612736,reserved:(0 bytes) 0; movedfromreserved: false,moveToReserveredfalse2023-01-05 18:26:54,158 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Invocation exception, caused by: com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host 1: Unable 
   to start instance due to Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying2023-01-05 18:26:54,158 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517 ctx-84952946) (logid:1c5b362f) Rethrow exception com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host 1: Unable to start instan
   ce due to Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying2023-01-05 18:26:54,158 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517) (logid:1c5b362f) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 99, job origin: 516
   2023-01-05 18:26:54,158 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517) (logid:1c5b362f) Unable to complete AsyncJobVO: {id:517, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: r
   O0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAY3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2FkRmFjdG9ySQAJdGhyZXNob2xkeHA_QAAAAAAADHcIAAAAEAAAAAF0AA5SZXN0YXJ0TmV0d29ya3QAP3JPMEFCWE55QUJGcVlYWmhMbXhoYm1jdVFtOXZiR1ZoYnMwZ2NvRFZuUHJ1QWdBQldnQUZkbUZzZFdWNGNBRXhw, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 279278805475902, completeMsid: null, lastUpdated: null, lastPolled: null,
  created: Thu Jan 05 18:26:35 CST 2023, removed: null}, job origin:516com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host 1: Unable to start instance due to Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1335)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
   	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
   	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   Caused by: com.cloud.utils.exception.ExecutionException: Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1299)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	... 18 more
   2023-01-05 18:26:54,175 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-4:ctx-bdb6d533 job-516/job-517) (logid:1c5b362f) Remove job-517 from job monitoring
   2023-01-05 18:26:54,195 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-4:ctx-25433880 job-516) (logid:1c5b362f) Unexpected exception while executing org.apache.cloudstack.api.command.admin.router.StartRouterCmd
   com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host 1: Unable to start instance due to Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1335)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
   	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
   	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
   	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
   	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
   	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
   	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   Caused by: com.cloud.utils.exception.ExecutionException: Unable to start  VM:4fcea489-51f8-4bd9-baef-65dc22828a94 due to error in finalizeStart, not retrying
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1299)
   	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	... 18 more
   2023-01-05 18:26:54,204 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-4:ctx-25433880 job-516) (logid:1c5b362f) Remove job-516 from job monitoring
   2023-01-05 18:26:54,355 DEBUG [c.c.a.ApiServlet] (qtp1747352992-40:ctx-131d1182) (logid:0ddf6efa) ===START===  192.168.0.103 -- GET  jobId=1c5b362f-6ddc-4ee0-b0f7-6b5b2c8f9dc3&command=queryAsyncJobResult&response=json
   2023-01-05 18:26:54,360 DEBUG [c.c.a.ApiServer] (qtp1747352992-40:ctx-131d1182 ctx-d7d071b3) (logid:0ddf6efa) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d"}
   ' is allowed to perform API calls: 0.0.0.0/0,::/02023-01-05 18:26:54,371 DEBUG [c.c.a.ApiServlet] (qtp1747352992-40:ctx-131d1182 ctx-d7d071b3) (logid:0ddf6efa) ===END===  192.168.0.103 -- GET  jobId=1c5b362f-6ddc-4ee0-b0f7-6b5b2c8f9dc3&command=queryAsyncJobResult&response=json
   2023-01-05 18:26:54,461 DEBUG [c.c.a.ApiServlet] (qtp1747352992-37:ctx-bff314b1) (logid:516f1d8c) ===START===  192.168.0.103 -- GET  listall=true&projectid=-1&id=4fcea489-51f8-4bd9-baef-65dc22828a94&page=1&pagesize=20&command=listRouters&response=json
   2023-01-05 18:26:54,466 DEBUG [c.c.a.ApiServer] (qtp1747352992-37:ctx-bff314b1 ctx-22ea9ae1) (logid:516f1d8c) CIDRs from which account 'Acct[2217c707-8817-11ed-9cb1-04421a1ae40d-admin] -- Account {"id": 2, "name": "admin", "uuid": "2217c707-8817-11ed-9cb1-04421a1ae40d"}
   ' is allowed to perform API calls: 0.0.0.0/0,::/0`
   Failed to start vr


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371744738

   The startup status is always up. This problem always occurs when creating an instance after uploading the iso image. How can I solve it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] fangyi1008 commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
fangyi1008 commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1371992416

   > > > @fangyi1008 您使用本地存储还是共享存储?
   > > > 日志显示没有适合该卷的存储池。
   > > > ```
   > > > 2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   > > > 2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   > > > 2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   > > > 2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   > > > 2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   > > > 2023-01-05 10:36:33,027 No suitable pools found
   > > > 2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   > > > ```
   > > 
   > > 
   > > The primary storage is local storage, and the secondary storage uses nfs。 In the global configuration, system.vm.use.local.storage is set to true
   > 
   > @fangyi1008 what's the storage type of service offering (used by the vm instance) ? it should be "local" as well.
   
   Select local when you need to customize the calculation scheme to select the disk size。
   thank you


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [cloudstack] weizhouapache commented on issue #7053: cant not instance

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #7053:
URL: https://github.com/apache/cloudstack/issues/7053#issuecomment-1372002028

   > > > > @fangyi1008 您使用本地存储还是共享存储?
   > > > > 日志显示没有适合该卷的存储池。
   > > > > ```
   > > > > 2023-01-05 10:36:33,020 Looking for suitable pools for this host under zone: 1, pod: 1, cluster: 1
   > > > > 2023-01-05 10:36:33,022 Checking suitable pools for volume (Id, Type): (70,ROOT)
   > > > > 2023-01-05 10:36:33,022 We need to allocate new storagepool for this volume
   > > > > 2023-01-05 10:36:33,023 Calling StoragePoolAllocators to find suitable pools
   > > > > 2023-01-05 10:36:33,027 No suitable pools found for volume: Vol[70|vm=90|ROOT] under cluster: 1
   > > > > 2023-01-05 10:36:33,027 No suitable pools found
   > > > > 2023-01-05 10:36:33,027 Cannot deploy to specified host, returning.
   > > > > ```
   > > > 
   > > > 
   > > > The primary storage is local storage, and the secondary storage uses nfs。 In the global configuration, system.vm.use.local.storage is set to true
   > > 
   > > 
   > > @fangyi1008 what's the storage type of service offering (used by the vm instance) ? it should be "local" as well.
   > > Another problem arises。
   > > Unable to start a VM [fdb11a9c-b45a-4a32-b6f9-c322a2358444] due to [Unable to create a deployment for VM instance {id: "98", name: "i-2-98-VM", uuid: "fdb11a9c-b45a-4a32-b6f9-c322a2358444", type="User"}].
   
   can you share more logs, please ?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org