You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Sailaja Mada <sa...@citrix.com> on 2013/05/16 13:49:01 UTC

[ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Hi,

With latest master , System VM's are failed to start with NPE. Is there any issue with recent fixes ? ( VMWARE)

2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl] (secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0, actual total: 9572, total with overprovisioning: 9572; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl] (secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0, total: 17166258176; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-05-16 17:12:20,524 WARN  [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Exception while trying to start secondary storage vm
java.lang.NullPointerException
        at com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityManagerImpl.java:684)
        at com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityManagerImpl.java:90)
        at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
        at com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachineManagerImpl.java:1251)
        at com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachineManagerImpl.java:660)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:817)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:547)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:540)
        at com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:262)
        at com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:687)
        at com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1303)
        at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123)
        at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50)
        at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
        at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
        at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
        at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-05-16 17:12:20,526 INFO  [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Unable to start secondary storage vm for standby capacity, secStorageVm vm Id : 4, will recycle it and start a new one
2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:null) VM is already stopped: VM[SecondaryStorageVm|s-4-VM]
2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl] (secstorage-1:null) VM state transitted from :Stopped to Expunging with event: ExpungeOperationvm's original host id: null new host id: null host id before state transition: null
2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]

Thanks,
Sailaja.M

Re: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Posted by Rohit Yadav <bh...@apache.org>.
Hi Abhi, can you share the workaround for fixing the issue we were having
for building systemvms for VMWare. I don't see any commit in
tools/appliance related to it, in case it's already fixed is the buildjob
on jenkins producing good enough VMWare systemvm templates?

Cheers.

On Thu, May 16, 2013 at 5:19 PM, Sailaja Mada <sa...@citrix.com>wrote:

> Hi,
>
> With latest master , System VM's are failed to start with NPE. Is there
> any issue with recent fixes ? ( VMWARE)
>
> 2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> (secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0,
> actual total: 9572, total with overprovisioning: 9572; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> (secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0,
> total: 17166258176; new used: 0,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2013-05-16 17:12:20,524 WARN
>  [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> Exception while trying to start secondary storage vm
> java.lang.NullPointerException
>         at
> com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityManagerImpl.java:684)
>         at
> com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityManagerImpl.java:90)
>         at
> com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
>         at
> com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachineManagerImpl.java:1251)
>         at
> com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachineManagerImpl.java:660)
>         at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:817)
>         at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:547)
>         at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:540)
>         at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:262)
>         at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:687)
>         at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1303)
>         at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123)
>         at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50)
>         at
> com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
>         at
> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>         at
> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>         at
> com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-05-16 17:12:20,526 INFO
>  [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Unable
> to start secondary storage vm for standby capacity, secStorageVm vm Id : 4,
> will recycle it and start a new one
> 2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (secstorage-1:null) VM is already stopped: VM[SecondaryStorageVm|s-4-VM]
> 2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl]
> (secstorage-1:null) VM state transitted from :Stopped to Expunging with
> event: ExpungeOperationvm's original host id: null new host id: null host
> id before state transition: null
> 2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]
>
> Thanks,
> Sailaja.M
>

RE: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Posted by Sailaja Mada <sa...@citrix.com>.
Thanks Kishan for the quick fix provided with https://issues.apache.org/jira/browse/CLOUDSTACK-2544 .

Regards,
Sailaja.M

________________________________________
From: Kishan Kavala
Sent: Thursday, May 16, 2013 6:35 PM
To: dev@cloudstack.apache.org
Cc: Sailaja Mada
Subject: RE: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

It happens only when Vm is staring for the first time irrespective of the hypervisor. This is due to 6d57393629bb93fbf0944e462c841f245b40f919.
Below check on CapacityManagerImpl.java:684 should change from

if (vm.getLastHostId().equals(vm.getHostId())) {

to

if (vm.getHostId().equals(vm.getLastHostId())) {

Last host Id is null when Vm starts for the 1st time.


> -----Original Message-----
> From: Likitha Shetty [mailto:likitha.shetty@citrix.com]
> Sent: Thursday, 16 May 2013 6:22 PM
> To: dev@cloudstack.apache.org
> Cc: Sailaja Mada
> Subject: RE: [ACS42] System VM's are failed to start with NPE with latest
> master(VMWARE)
>
> I saw the same issue with Xen.
>
> Thanks,
> Likitha
>
> >-----Original Message-----
> >From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >Sent: Thursday, May 16, 2013 5:54 PM
> >To: dev@cloudstack.apache.org
> >Cc: Sailaja Mada
> >Subject: Re: [ACS42] System VM's are failed to start with NPE with
> >latest
> >master(VMWARE)
> >
> >Is this only for vmware ? Did you try other hypervisors since it looks
> >generic enough ?
> >
> >On 16/05/13 5:19 PM, "Sailaja Mada" <sa...@citrix.com> wrote:
> >
> >>Hi,
> >>
> >>With latest master , System VM's are failed to start with NPE. Is
> >>there any issue with recent fixes ? ( VMWARE)
> >>
> >>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> >>(secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0,
> >>actual total: 9572, total with overprovisioning: 9572; new used:
> >>0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> >>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> >>(secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0,
> >>total: 17166258176; new used: 0,reserved:0; movedfromreserved:
> >>false,moveToReserveredfalse
> >>2013-05-16 17:12:20,524 WARN
> >>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> >>Exception while trying to start secondary storage vm
> >>java.lang.NullPointerException
> >>        at
> >>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capac
> i
> >>t
> >>yMa
> >>nagerImpl.java:684)
> >>        at
> >>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capac
> i
> >>t
> >>yMa
> >>nagerImpl.java:90)
> >>        at
> >>com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachine
> Ma
> >>n
> >>age
> >>rImpl.java:1251)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachine
> Mana
> >ge
> >>rIm
> >>pl.java:660)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachine
> Mana
> >g
> >>erI
> >>mpl.java:817)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerI
> mpl.
> >>jav
> >>a:547)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerI
> mpl.
> >>jav
> >>a:540)
> >>        at
> >>com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecSto
> rag
> >>e
> >>Vm(
> >>SecondaryStorageManagerImpl.java:262)
> >>        at
> >>com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacit
> y(
> >>S
> >>eco
> >>ndaryStorageManagerImpl.java:687)
> >>        at
> >>com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool
> (Sec
> >>o
> >>nda
> >>ryStorageManagerImpl.java:1303)
> >>        at
> >>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(
> Premi
> >u
> >>mSe
> >>condaryStorageManagerImpl.java:123)
> >>        at
> >>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(
> Premi
> >u
> >>mSe
> >>condaryStorageManagerImpl.java:50)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.j
> ava:10
> >4)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner
> .java:
> >33)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanne
> r.java
> >:81)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java
> :72)
> >>        at
> >>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >>        at
> >>java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:
> 351)
> >>        at
> >>java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> >>        at
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
> a
> >>c
> >>ces
> >>s$201(ScheduledThreadPoolExecutor.java:165)
> >>        at
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
> r
> >>u
> >>n(S
> >>cheduledThreadPoolExecutor.java:267)
> >>        at
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:
> >>1110)
> >>        at
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> >>j
> >>ava
> >>:603)
> >>        at java.lang.Thread.run(Thread.java:679)
> >>2013-05-16 17:12:20,526 INFO
> >>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> >>Unable to start secondary storage vm for standby capacity,
> >>secStorageVm vm Id : 4, will recycle it and start a new one
> >>2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >>(secstorage-1:null) VM is already stopped:
> >>VM[SecondaryStorageVm|s-4-VM]
> >>2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl]
> >>(secstorage-1:null) VM state transitted from :Stopped to Expunging
> >>with
> >>event: ExpungeOperationvm's original host id: null new host id: null
> >>host id before state transition: null
> >>2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >>(secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]
> >>
> >>Thanks,
> >>Sailaja.M


RE: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Posted by Kishan Kavala <Ki...@citrix.com>.
It happens only when Vm is staring for the first time irrespective of the hypervisor. This is due to 6d57393629bb93fbf0944e462c841f245b40f919.
Below check on CapacityManagerImpl.java:684 should change from

if (vm.getLastHostId().equals(vm.getHostId())) {

to 

if (vm.getHostId().equals(vm.getLastHostId())) {

Last host Id is null when Vm starts for the 1st time.


> -----Original Message-----
> From: Likitha Shetty [mailto:likitha.shetty@citrix.com]
> Sent: Thursday, 16 May 2013 6:22 PM
> To: dev@cloudstack.apache.org
> Cc: Sailaja Mada
> Subject: RE: [ACS42] System VM's are failed to start with NPE with latest
> master(VMWARE)
> 
> I saw the same issue with Xen.
> 
> Thanks,
> Likitha
> 
> >-----Original Message-----
> >From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >Sent: Thursday, May 16, 2013 5:54 PM
> >To: dev@cloudstack.apache.org
> >Cc: Sailaja Mada
> >Subject: Re: [ACS42] System VM's are failed to start with NPE with
> >latest
> >master(VMWARE)
> >
> >Is this only for vmware ? Did you try other hypervisors since it looks
> >generic enough ?
> >
> >On 16/05/13 5:19 PM, "Sailaja Mada" <sa...@citrix.com> wrote:
> >
> >>Hi,
> >>
> >>With latest master , System VM's are failed to start with NPE. Is
> >>there any issue with recent fixes ? ( VMWARE)
> >>
> >>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> >>(secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0,
> >>actual total: 9572, total with overprovisioning: 9572; new used:
> >>0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> >>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> >>(secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0,
> >>total: 17166258176; new used: 0,reserved:0; movedfromreserved:
> >>false,moveToReserveredfalse
> >>2013-05-16 17:12:20,524 WARN
> >>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> >>Exception while trying to start secondary storage vm
> >>java.lang.NullPointerException
> >>        at
> >>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capac
> i
> >>t
> >>yMa
> >>nagerImpl.java:684)
> >>        at
> >>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capac
> i
> >>t
> >>yMa
> >>nagerImpl.java:90)
> >>        at
> >>com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachine
> Ma
> >>n
> >>age
> >>rImpl.java:1251)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachine
> Mana
> >ge
> >>rIm
> >>pl.java:660)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachine
> Mana
> >g
> >>erI
> >>mpl.java:817)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerI
> mpl.
> >>jav
> >>a:547)
> >>        at
> >>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerI
> mpl.
> >>jav
> >>a:540)
> >>        at
> >>com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecSto
> rag
> >>e
> >>Vm(
> >>SecondaryStorageManagerImpl.java:262)
> >>        at
> >>com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacit
> y(
> >>S
> >>eco
> >>ndaryStorageManagerImpl.java:687)
> >>        at
> >>com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool
> (Sec
> >>o
> >>nda
> >>ryStorageManagerImpl.java:1303)
> >>        at
> >>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(
> Premi
> >u
> >>mSe
> >>condaryStorageManagerImpl.java:123)
> >>        at
> >>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(
> Premi
> >u
> >>mSe
> >>condaryStorageManagerImpl.java:50)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.j
> ava:10
> >4)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner
> .java:
> >33)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanne
> r.java
> >:81)
> >>        at
> >>com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java
> :72)
> >>        at
> >>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >>        at
> >>java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:
> 351)
> >>        at
> >>java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> >>        at
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
> a
> >>c
> >>ces
> >>s$201(ScheduledThreadPoolExecutor.java:165)
> >>        at
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
> r
> >>u
> >>n(S
> >>cheduledThreadPoolExecutor.java:267)
> >>        at
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:
> >>1110)
> >>        at
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> >>j
> >>ava
> >>:603)
> >>        at java.lang.Thread.run(Thread.java:679)
> >>2013-05-16 17:12:20,526 INFO
> >>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> >>Unable to start secondary storage vm for standby capacity,
> >>secStorageVm vm Id : 4, will recycle it and start a new one
> >>2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >>(secstorage-1:null) VM is already stopped:
> >>VM[SecondaryStorageVm|s-4-VM]
> >>2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl]
> >>(secstorage-1:null) VM state transitted from :Stopped to Expunging
> >>with
> >>event: ExpungeOperationvm's original host id: null new host id: null
> >>host id before state transition: null
> >>2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >>(secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]
> >>
> >>Thanks,
> >>Sailaja.M


RE: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Posted by Likitha Shetty <li...@citrix.com>.
I saw the same issue with Xen.

Thanks,
Likitha

>-----Original Message-----
>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>Sent: Thursday, May 16, 2013 5:54 PM
>To: dev@cloudstack.apache.org
>Cc: Sailaja Mada
>Subject: Re: [ACS42] System VM's are failed to start with NPE with latest
>master(VMWARE)
>
>Is this only for vmware ? Did you try other hypervisors since it looks generic
>enough ?
>
>On 16/05/13 5:19 PM, "Sailaja Mada" <sa...@citrix.com> wrote:
>
>>Hi,
>>
>>With latest master , System VM's are failed to start with NPE. Is there
>>any issue with recent fixes ? ( VMWARE)
>>
>>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
>>(secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0,
>>actual total: 9572, total with overprovisioning: 9572; new used:
>>0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
>>(secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0,
>>total: 17166258176; new used: 0,reserved:0; movedfromreserved:
>>false,moveToReserveredfalse
>>2013-05-16 17:12:20,524 WARN
>>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
>>Exception while trying to start secondary storage vm
>>java.lang.NullPointerException
>>        at
>>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capacit
>>yMa
>>nagerImpl.java:684)
>>        at
>>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capacit
>>yMa
>>nagerImpl.java:90)
>>        at
>>com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
>>        at
>>com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachineMan
>>age
>>rImpl.java:1251)
>>        at
>>com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachineMana
>ge
>>rIm
>>pl.java:660)
>>        at
>>com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineMana
>g
>>erI
>>mpl.java:817)
>>        at
>>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.
>>jav
>>a:547)
>>        at
>>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.
>>jav
>>a:540)
>>        at
>>com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorage
>>Vm(
>>SecondaryStorageManagerImpl.java:262)
>>        at
>>com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(S
>>eco
>>ndaryStorageManagerImpl.java:687)
>>        at
>>com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(Seco
>>nda
>>ryStorageManagerImpl.java:1303)
>>        at
>>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(Premi
>u
>>mSe
>>condaryStorageManagerImpl.java:123)
>>        at
>>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(Premi
>u
>>mSe
>>condaryStorageManagerImpl.java:50)
>>        at
>>com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:10
>4)
>>        at
>>com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:
>33)
>>        at
>>com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java
>:81)
>>        at
>>com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>>        at
>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>        at
>>java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>>        at
>>java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>>        at
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac
>>ces
>>s$201(ScheduledThreadPoolExecutor.java:165)
>>        at
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru
>>n(S
>>cheduledThreadPoolExecutor.java:267)
>>        at
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>>1110)
>>        at
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>ava
>>:603)
>>        at java.lang.Thread.run(Thread.java:679)
>>2013-05-16 17:12:20,526 INFO
>>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
>>Unable to start secondary storage vm for standby capacity, secStorageVm
>>vm Id : 4, will recycle it and start a new one
>>2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>(secstorage-1:null) VM is already stopped:
>>VM[SecondaryStorageVm|s-4-VM]
>>2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl]
>>(secstorage-1:null) VM state transitted from :Stopped to Expunging with
>>event: ExpungeOperationvm's original host id: null new host id: null
>>host id before state transition: null
>>2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>(secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]
>>
>>Thanks,
>>Sailaja.M


RE: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Posted by Rajesh Battala <ra...@citrix.com>.
Am using new system vm template. Am on latest master. Its working fine for me.

> -----Original Message-----
> From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> Sent: Thursday, May 16, 2013 5:46 PM
> To: dev@cloudstack.apache.org
> Cc: Sailaja Mada
> Subject: Re: [ACS42] System VM's are failed to start with NPE with latest
> master(VMWARE)
> 
> Is this only for vmware ? Did you try other hypervisors since it looks generic
> enough ?
> 
> On 16/05/13 5:19 PM, "Sailaja Mada" <sa...@citrix.com> wrote:
> 
> >Hi,
> >
> >With latest master , System VM's are failed to start with NPE. Is there
> >any issue with recent fixes ? ( VMWARE)
> >
> >2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> >(secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0,
> >actual total: 9572, total with overprovisioning: 9572; new used:
> >0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> >2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
> >(secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0,
> >total: 17166258176; new used: 0,reserved:0; movedfromreserved:
> >false,moveToReserveredfalse
> >2013-05-16 17:12:20,524 WARN
> >[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> >Exception while trying to start secondary storage vm
> >java.lang.NullPointerException
> >        at
> >com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capacit
> >yMa
> >nagerImpl.java:684)
> >        at
> >com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(Capacit
> >yMa
> >nagerImpl.java:90)
> >        at
> >com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
> >        at
> >com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachine
> Man
> >age
> >rImpl.java:1251)
> >        at
> >com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachineMa
> nage
> >rIm
> >pl.java:660)
> >        at
> >com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineMa
> nag
> >erI
> >mpl.java:817)
> >        at
> >com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerIm
> pl.
> >jav
> >a:547)
> >        at
> >com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerIm
> pl.
> >jav
> >a:540)
> >        at
> >com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorag
> e
> >Vm(
> >SecondaryStorageManagerImpl.java:262)
> >        at
> >com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(
> S
> >eco
> >ndaryStorageManagerImpl.java:687)
> >        at
> >com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(S
> eco
> >nda
> >ryStorageManagerImpl.java:1303)
> >        at
> >com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(Pre
> miu
> >mSe
> >condaryStorageManagerImpl.java:123)
> >        at
> >com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(Pre
> miu
> >mSe
> >condaryStorageManagerImpl.java:50)
> >        at
> >com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:
> 104)
> >        at
> >com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.ja
> va:33)
> >        at
> >com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.ja
> va:81)
> >        at
> >com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
> >        at
> >java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >        at
> >java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:35
> 1)
> >        at
> >java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> >        at
> >java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac
> >ces
> >s$201(ScheduledThreadPoolExecutor.java:165)
> >        at
> >java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru
> >n(S
> >cheduledThreadPoolExecutor.java:267)
> >        at
> >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
> va:
> >1110)
> >        at
> >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
> >ava
> >:603)
> >        at java.lang.Thread.run(Thread.java:679)
> >2013-05-16 17:12:20,526 INFO
> >[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> >Unable to start secondary storage vm for standby capacity, secStorageVm
> >vm Id : 4, will recycle it and start a new one
> >2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >(secstorage-1:null) VM is already stopped:
> >VM[SecondaryStorageVm|s-4-VM]
> >2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl]
> >(secstorage-1:null) VM state transitted from :Stopped to Expunging with
> >event: ExpungeOperationvm's original host id: null new host id: null
> >host id before state transition: null
> >2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >(secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]
> >
> >Thanks,
> >Sailaja.M


Re: [ACS42] System VM's are failed to start with NPE with latest master(VMWARE)

Posted by Nitin Mehta <Ni...@citrix.com>.
Is this only for vmware ? Did you try other hypervisors since it looks
generic enough ?

On 16/05/13 5:19 PM, "Sailaja Mada" <sa...@citrix.com> wrote:

>Hi,
>
>With latest master , System VM's are failed to start with NPE. Is there
>any issue with recent fixes ? ( VMWARE)
>
>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
>(secstorage-1:null) release cpu from host: 1, old used: 0,reserved: 0,
>actual total: 9572, total with overprovisioning: 9572; new used:
>0,reserved:0; movedfromreserved: false,moveToReserveredfalse
>2013-05-16 17:12:20,523 DEBUG [cloud.capacity.CapacityManagerImpl]
>(secstorage-1:null) release mem from host: 1, old used: 0,reserved: 0,
>total: 17166258176; new used: 0,reserved:0; movedfromreserved:
>false,moveToReserveredfalse
>2013-05-16 17:12:20,524 WARN
>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
>Exception while trying to start secondary storage vm
>java.lang.NullPointerException
>        at 
>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityMa
>nagerImpl.java:684)
>        at 
>com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityMa
>nagerImpl.java:90)
>        at 
>com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
>        at 
>com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachineManage
>rImpl.java:1251)
>        at 
>com.cloud.vm.VirtualMachineManagerImpl.changeState(VirtualMachineManagerIm
>pl.java:660)
>        at 
>com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerI
>mpl.java:817)
>        at 
>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.jav
>a:547)
>        at 
>com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.jav
>a:540)
>        at 
>com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(
>SecondaryStorageManagerImpl.java:262)
>        at 
>com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(Seco
>ndaryStorageManagerImpl.java:687)
>        at 
>com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(Seconda
>ryStorageManagerImpl.java:1303)
>        at 
>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSe
>condaryStorageManagerImpl.java:123)
>        at 
>com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSe
>condaryStorageManagerImpl.java:50)
>        at 
>com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
>        at 
>com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>        at 
>com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>        at 
>com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>        at 
>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>        at 
>java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>        at 
>java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>        at 
>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acces
>s$201(ScheduledThreadPoolExecutor.java:165)
>        at 
>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(S
>cheduledThreadPoolExecutor.java:267)
>        at 
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1110)
>        at 
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:603)
>        at java.lang.Thread.run(Thread.java:679)
>2013-05-16 17:12:20,526 INFO
>[storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
>Unable to start secondary storage vm for standby capacity, secStorageVm
>vm Id : 4, will recycle it and start a new one
>2013-05-16 17:12:20,529 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>(secstorage-1:null) VM is already stopped: VM[SecondaryStorageVm|s-4-VM]
>2013-05-16 17:12:20,537 DEBUG [cloud.capacity.CapacityManagerImpl]
>(secstorage-1:null) VM state transitted from :Stopped to Expunging with
>event: ExpungeOperationvm's original host id: null new host id: null host
>id before state transition: null
>2013-05-16 17:12:20,540 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>(secstorage-1:null) Destroying vm VM[SecondaryStorageVm|s-4-VM]
>
>Thanks,
>Sailaja.M