You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users-cn@cloudstack.apache.org by 王磊 <wa...@smartdot.com> on 2013/11/20 04:41:05 UTC

回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除

下面为日志,麻烦了

2013-11-20 11:03:38,338 INFO  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Found 6 vms to expunge.
2013-11-20 11:03:38,338 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|windows-2008-R2-X64] but the state is Expunging
2013-11-20 11:03:38,345 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 5 new host id: null host id before state transition: null
2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|windows-2008-R2-X64]
2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS
2013-11-20 11:03:38,345 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 50
2013-11-20 11:03:38,348 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 50
2013-11-20 11:03:38,352 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name =  cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165
2013-11-20 11:03:38,352 WARN  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|windows-2008-R2-X64]
com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 3
 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60)
 at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45)
 at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89)
 at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581)
 at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145)
 at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479)
 at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600)
 at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:701)
2013-11-20 11:03:38,353 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|win2008-1] but the state is Expunging
2013-11-20 11:03:38,365 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 7 new host id: null host id before state transition: null
2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|win2008-1]
2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS
2013-11-20 11:03:38,365 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 52
2013-11-20 11:03:38,368 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 52
2013-11-20 11:03:38,371 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name =  cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165
2013-11-20 11:03:38,372 WARN  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|win2008-1]
com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 5
 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60)
 at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45)
 at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89)
 at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581)
 at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145)
 at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479)
 at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600)
 at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:701) 



发件人:tanthalas <ta...@hotmail.com>
发送时间:2013-11-20 10:29
主题:Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除
收件人:"users-cn"<us...@cloudstack.apache.org>
抄送:


因为你先违规操作,先删除的主机,但你给的信息还不够,不够判断。 
这个集群所有主机都删除了吗?还只是随意删除一台? 
但是数据卷不能删除,我感觉问题比较大,log里肯定有对应信息的。 

   
刘宇超  Richard Liu 




发件人: 王磊  
发送时间: 2013-11-20  10:06:05  
收件人: users-cn  
抄送:  
主题: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除  
  
expunge.delay和expunge.interval我已经全都改成600了 
发件人:Du Jun <dj...@gmail.com> 
发送时间:2013-11-20 09:19 
主题:Re: 回复: Re: 实例一直是Expunging状态,无法删除 
收件人:"users-cn"<us...@cloudstack.apache.org> 
抄送: 
实际上你已经删除实例了,只是CS会将删除的实例保存一段时间。你可以在global  
settings里面搜索expuge关键字,设置expuge的时间,比如120。那么,过2分钟就会在UI上面移除虚拟机。  
在 2013年11月19日下午4:23,toudsf <18...@163.com>写道:  
> 到数据库中,有一个表instance吧好像。把状态变为Stopped然后从画面进行删除。  
>  
> 2013-11-19  
>  
>  
>  
> toudsf  
>  
>  
>  
> 发件人:"王磊" <wa...@smartdot.com>  
> 发送时间:2013-11-19 16:18  
> 主题:回复: Re: 实例一直是Expunging状态,无法删除  
> 收件人:"users-cn"<us...@cloudstack.apache.org>  
> 抄送:  
>  
> 没有错误提示,页面显示删除成功,但就是无法彻底删除,一直在Expunging,从实例点进去也一直在灰色页面转圈圈没有任何显示  
>  
> 实例对应的ip已经释放  
>  
> 实例对应的存储也无法删除  
>  
>  
>  
>  
>  
>  
> 发件人:Wei ZHOU <us...@gmail.com>  
> 发送时间:2013-11-19 16:04  
> 主题:Re: 实例一直是Expunging状态,无法删除  
> 收件人:"users-cn"<us...@cloudstack.apache.org>  
> 抄送:  
>  
> 有什么错误提示吗?  
> 直接改也问题不大  
>  
>  
> 在 2013年11月19日上午9:02,王磊 <wa...@smartdot.com>写道:  
>  
> > 没有正常删除主机(直接把主机格式化重做系统,后在管理端删除的主机和实例),导致实例一直是Expunging状态,无法删除  
> >  
> > expunge.delay和expunge.interval都已经改成了600,其他实例都可以正常删除。  
> >  
> > 应该怎么做,直接到数据库里操作吗?  
>  

回复: Re: 回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除

Posted by 王磊 <wa...@smartdot.com>.
我先在数据库里把卷删掉了,然后那些处于Expunging状态的实例自己就消失了

谢谢各位了!
-------------------------------------------------------------------------------------------------------------------------------



发件人:tanthalas <ta...@hotmail.com>
发送时间:2013-11-20 12:17
主题:Re: 回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除
收件人:"users-cn"<us...@cloudstack.apache.org>
抄送:


就怕的是这个,错误的顺序删除了主机,之后的问题就不好解决了。 
集群的主机都删除了,但存储还在?想去存储里删除卷的时候,系统也在不停尝试expung,其实还是需要主机的命令,以及连接。 
你试试加个主机吧这个集群恢复看看。 
实在不行再改数据库。删除vm,数据卷的操作不是问题,但删除存储节点可能会有问题。 

刘宇超  Richard Liu 




发件人: 王磊  
发送时间: 2013-11-20  11:41:56  
收件人: users-cn  
抄送:  
主题: 回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除  
  
下面为日志,麻烦了 
2013-11-20 11:03:38,338 INFO  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Found 6 vms to expunge. 
2013-11-20 11:03:38,338 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|windows-2008-R2-X64] but the state is Expunging 
2013-11-20 11:03:38,345 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 5 new host id: null host id before state transition: null 
2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|windows-2008-R2-X64] 
2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS 
2013-11-20 11:03:38,345 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 50 
2013-11-20 11:03:38,348 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 50 
2013-11-20 11:03:38,352 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name =  cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165 
2013-11-20 11:03:38,352 WARN  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|windows-2008-R2-X64] 
com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 3 
 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60) 
 at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45) 
 at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89) 
 at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581) 
 at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145) 
 at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) 
 at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479) 
 at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600) 
 at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769) 
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) 
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) 
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) 
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) 
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
 at java.lang.Thread.run(Thread.java:701) 
2013-11-20 11:03:38,353 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|win2008-1] but the state is Expunging 
2013-11-20 11:03:38,365 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 7 new host id: null host id before state transition: null 
2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|win2008-1] 
2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS 
2013-11-20 11:03:38,365 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 52 
2013-11-20 11:03:38,368 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 52 
2013-11-20 11:03:38,371 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name =  cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165 
2013-11-20 11:03:38,372 WARN  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|win2008-1] 
com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 5 
 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60) 
 at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45) 
 at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89) 
 at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581) 
 at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145) 
 at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) 
 at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479) 
 at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600) 
 at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769) 
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) 
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) 
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) 
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) 
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
 at java.lang.Thread.run(Thread.java:701)  
发件人:tanthalas <ta...@hotmail.com> 
发送时间:2013-11-20 10:29 
主题:Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除 
收件人:"users-cn"<us...@cloudstack.apache.org> 
抄送: 
因为你先违规操作,先删除的主机,但你给的信息还不够,不够判断。  
这个集群所有主机都删除了吗?还只是随意删除一台?  
但是数据卷不能删除,我感觉问题比较大,log里肯定有对应信息的。  
    
刘宇超  Richard Liu  
发件人: 王磊   
发送时间: 2013-11-20  10:06:05   
收件人: users-cn   
抄送:   
主题: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除   
   
expunge.delay和expunge.interval我已经全都改成600了  
发件人:Du Jun <dj...@gmail.com>  
发送时间:2013-11-20 09:19  
主题:Re: 回复: Re: 实例一直是Expunging状态,无法删除  
收件人:"users-cn"<us...@cloudstack.apache.org>  
抄送:  
实际上你已经删除实例了,只是CS会将删除的实例保存一段时间。你可以在global   
settings里面搜索expuge关键字,设置expuge的时间,比如120。那么,过2分钟就会在UI上面移除虚拟机。   
在 2013年11月19日下午4:23,toudsf <18...@163.com>写道:   
> 到数据库中,有一个表instance吧好像。把状态变为Stopped然后从画面进行删除。   
>   
> 2013-11-19   
>   
>   
>   
> toudsf   
>   
>   
>   
> 发件人:"王磊" <wa...@smartdot.com>   
> 发送时间:2013-11-19 16:18   
> 主题:回复: Re: 实例一直是Expunging状态,无法删除   
> 收件人:"users-cn"<us...@cloudstack.apache.org>   
> 抄送:   
>   
> 没有错误提示,页面显示删除成功,但就是无法彻底删除,一直在Expunging,从实例点进去也一直在灰色页面转圈圈没有任何显示   
>   
> 实例对应的ip已经释放   
>   
> 实例对应的存储也无法删除   
>   
>   
>   
>   
>   
>   
> 发件人:Wei ZHOU <us...@gmail.com>   
> 发送时间:2013-11-19 16:04   
> 主题:Re: 实例一直是Expunging状态,无法删除   
> 收件人:"users-cn"<us...@cloudstack.apache.org>   
> 抄送:   
>   
> 有什么错误提示吗?   
> 直接改也问题不大   
>   
>   
> 在 2013年11月19日上午9:02,王磊 <wa...@smartdot.com>写道:   
>   
> > 没有正常删除主机(直接把主机格式化重做系统,后在管理端删除的主机和实例),导致实例一直是Expunging状态,无法删除   
> >   
> > expunge.delay和expunge.interval都已经改成了600,其他实例都可以正常删除。   
> >   
> > 应该怎么做,直接到数据库里操作吗?   
>   

Re: 回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除

Posted by tanthalas <ta...@hotmail.com>.
就怕的是这个,错误的顺序删除了主机,之后的问题就不好解决了。
集群的主机都删除了,但存储还在?想去存储里删除卷的时候,系统也在不停尝试expung,其实还是需要主机的命令,以及连接。
你试试加个主机吧这个集群恢复看看。
实在不行再改数据库。删除vm,数据卷的操作不是问题,但删除存储节点可能会有问题。

刘宇超  Richard Liu




发件人: 王磊 
发送时间: 2013-11-20  11:41:56 
收件人: users-cn 
抄送: 
主题: 回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除 
 
下面为日志,麻烦了
2013-11-20 11:03:38,338 INFO  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Found 6 vms to expunge.
2013-11-20 11:03:38,338 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|windows-2008-R2-X64] but the state is Expunging
2013-11-20 11:03:38,345 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 5 new host id: null host id before state transition: null
2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|windows-2008-R2-X64]
2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS
2013-11-20 11:03:38,345 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 50
2013-11-20 11:03:38,348 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 50
2013-11-20 11:03:38,352 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name =  cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165
2013-11-20 11:03:38,352 WARN  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|windows-2008-R2-X64]
com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 3
 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60)
 at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45)
 at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89)
 at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581)
 at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145)
 at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479)
 at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600)
 at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:701)
2013-11-20 11:03:38,353 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|win2008-1] but the state is Expunging
2013-11-20 11:03:38,365 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 7 new host id: null host id before state transition: null
2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|win2008-1]
2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS
2013-11-20 11:03:38,365 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 52
2013-11-20 11:03:38,368 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 52
2013-11-20 11:03:38,371 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name =  cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165
2013-11-20 11:03:38,372 WARN  [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|win2008-1]
com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 5
 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60)
 at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45)
 at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89)
 at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581)
 at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145)
 at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479)
 at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600)
 at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:701) 
发件人:tanthalas <ta...@hotmail.com>
发送时间:2013-11-20 10:29
主题:Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除
收件人:"users-cn"<us...@cloudstack.apache.org>
抄送:
因为你先违规操作,先删除的主机,但你给的信息还不够,不够判断。 
这个集群所有主机都删除了吗?还只是随意删除一台? 
但是数据卷不能删除,我感觉问题比较大,log里肯定有对应信息的。 
   
刘宇超  Richard Liu 
发件人: 王磊  
发送时间: 2013-11-20  10:06:05  
收件人: users-cn  
抄送:  
主题: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除  
  
expunge.delay和expunge.interval我已经全都改成600了 
发件人:Du Jun <dj...@gmail.com> 
发送时间:2013-11-20 09:19 
主题:Re: 回复: Re: 实例一直是Expunging状态,无法删除 
收件人:"users-cn"<us...@cloudstack.apache.org> 
抄送: 
实际上你已经删除实例了,只是CS会将删除的实例保存一段时间。你可以在global  
settings里面搜索expuge关键字,设置expuge的时间,比如120。那么,过2分钟就会在UI上面移除虚拟机。  
在 2013年11月19日下午4:23,toudsf <18...@163.com>写道:  
> 到数据库中,有一个表instance吧好像。把状态变为Stopped然后从画面进行删除。  
>  
> 2013-11-19  
>  
>  
>  
> toudsf  
>  
>  
>  
> 发件人:"王磊" <wa...@smartdot.com>  
> 发送时间:2013-11-19 16:18  
> 主题:回复: Re: 实例一直是Expunging状态,无法删除  
> 收件人:"users-cn"<us...@cloudstack.apache.org>  
> 抄送:  
>  
> 没有错误提示,页面显示删除成功,但就是无法彻底删除,一直在Expunging,从实例点进去也一直在灰色页面转圈圈没有任何显示  
>  
> 实例对应的ip已经释放  
>  
> 实例对应的存储也无法删除  
>  
>  
>  
>  
>  
>  
> 发件人:Wei ZHOU <us...@gmail.com>  
> 发送时间:2013-11-19 16:04  
> 主题:Re: 实例一直是Expunging状态,无法删除  
> 收件人:"users-cn"<us...@cloudstack.apache.org>  
> 抄送:  
>  
> 有什么错误提示吗?  
> 直接改也问题不大  
>  
>  
> 在 2013年11月19日上午9:02,王磊 <wa...@smartdot.com>写道:  
>  
> > 没有正常删除主机(直接把主机格式化重做系统,后在管理端删除的主机和实例),导致实例一直是Expunging状态,无法删除  
> >  
> > expunge.delay和expunge.interval都已经改成了600,其他实例都可以正常删除。  
> >  
> > 应该怎么做,直接到数据库里操作吗?  
>