System VM's hanging

[Sean Truman <st...@gmail.com>]

All,

My system VM's are hanging at GRUB loading stage2.. using KVM as hypervisor
any idea why this is happening?

v/r
Sean

Re: System VM's hanging

[Sean Truman <st...@gmail.com>]

I figured it out, I was using shared mount points because I was trying to
use NFS over RDMA (Infiniband, it doesn't support locking :( ).  I switched
the primary over to NFS over Infiniband but using IPoIB (Ethernet instead
of RDMA).  Now the System VM's are booting correctly. Thanks for your help.

v/r
Sean


On Mon, May 6, 2013 at 12:25 PM, Musayev, Ilya <im...@webmd.net> wrote:

> Sean,
>
> Please see response in line.
>
> > I am not seeing any splash screen, I see the Seabios screens and then
> the PIX
> > boot line, then GRUB loading stage2.. and the KVM process eats 100% of
> one
> > of the cores for about 5 minutes, then the management server tries to
> > restart.  I am using CS 4.0.2
> >
> > 0) acton-systemvm-02062012.qcow2.bz2
> This image has been used countless times by many folks, perhaps something
> is unique in your environment that could cause this problem?
>
> > 1) I will try this, I ran qemu-img check on all the image files and they
> all are
> > fine (I did a virsh dumpxml s-1-vm and looked at the disk it was
> referencing),
> > I have also reinstalled SystemVM as per installation directions.
> > 2)  I cannot even get it to let me get to a point where it will let me
> edit grub
> > boot parameters.
>
>
> > 3)  Like I said above the System VM doesn't even show me a splash screen.
> You don't need grub to function in this case, since you are booting off
> linux rescue cd to repair grub.
>
> >
> > I have reinstalled management and system templates and am trying it
> again.
> >  I will also try to run the system VM's by command line and see if it
> works
> > from command line. I had everything working with basic networking, as
> soon
> > as I switch it over to advance networking (using VLAN) is when this
> started
> > happening.
> The issue between switching from basic to advanced and router VM makes no
> sense. The only thing that would change is the metadata details vm will use
> to boot with, but that comes after GRUB.
>
> >
> > v/r
> > Sean
> >
> >
> > On Mon, May 6, 2013 at 11:36 AM, Musayev, Ilya <im...@webmd.net>
> > wrote:
> >
> > > Sean,
> > >
> > > Specifically what versions of SVM are you running? Do you see a splash
> > > screen and when you press enter or wait for time out - for this issue
> > > to occur?
> > >
> > > 0) if splash screen is seen and you can make changes to boot linux,
> > > remove the "quiet" flag from initrd line to see if you can get more
> > > meaningful errors
> > >
> > > 1) I would suggest to change the storage controllers to see if it
> > > makes any difference.
> > >
> > > 2) I've seen in past " GRUB loading stage2" error would occur when
> > > grub.conf reference initrd or kernel version that no longer exists
> > > (filename mismatch). Go into the edit mode, remove the version in
> > > front of initrd and vmlinuz and try to use auto-completion (TAB-TAB)
> > > to complete the filename and boot again.
> > >
> > > 3) If nothing helps,  I would have tried to boot it into a single user
> > > mode with linux rescue disk and reinstall grub - just to confirm if
> > > its grub related or something else.
> > >
> > > Let me know how it goes,
> > >
> > > Regards
> > > ilya
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Sean Truman [mailto:struman@gmail.com]
> > > > Sent: Monday, May 06, 2013 12:16 PM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: System VM's hanging
> > > >
> > > > All,
> > > >
> > > > My system VM's are hanging at GRUB loading stage2.. using KVM as
> > > > hypervisor any idea why this is happening?
> > > >
> > > > v/r
> > > > Sean
> > >
> > >
>
>

RE: System VM's hanging

["Musayev, Ilya" <im...@webmd.net>]

Sean,

Please see response in line.

> I am not seeing any splash screen, I see the Seabios screens and then the PIX
> boot line, then GRUB loading stage2.. and the KVM process eats 100% of one
> of the cores for about 5 minutes, then the management server tries to
> restart.  I am using CS 4.0.2
> 
> 0) acton-systemvm-02062012.qcow2.bz2
This image has been used countless times by many folks, perhaps something is unique in your environment that could cause this problem?

> 1) I will try this, I ran qemu-img check on all the image files and they all are
> fine (I did a virsh dumpxml s-1-vm and looked at the disk it was referencing),
> I have also reinstalled SystemVM as per installation directions.
> 2)  I cannot even get it to let me get to a point where it will let me edit grub
> boot parameters.


> 3)  Like I said above the System VM doesn't even show me a splash screen.
You don't need grub to function in this case, since you are booting off linux rescue cd to repair grub.

> 
> I have reinstalled management and system templates and am trying it again.
>  I will also try to run the system VM's by command line and see if it works
> from command line. I had everything working with basic networking, as soon
> as I switch it over to advance networking (using VLAN) is when this started
> happening.
The issue between switching from basic to advanced and router VM makes no sense. The only thing that would change is the metadata details vm will use to boot with, but that comes after GRUB. 

> 
> v/r
> Sean
> 
> 
> On Mon, May 6, 2013 at 11:36 AM, Musayev, Ilya <im...@webmd.net>
> wrote:
> 
> > Sean,
> >
> > Specifically what versions of SVM are you running? Do you see a splash
> > screen and when you press enter or wait for time out - for this issue
> > to occur?
> >
> > 0) if splash screen is seen and you can make changes to boot linux,
> > remove the "quiet" flag from initrd line to see if you can get more
> > meaningful errors
> >
> > 1) I would suggest to change the storage controllers to see if it
> > makes any difference.
> >
> > 2) I've seen in past " GRUB loading stage2" error would occur when
> > grub.conf reference initrd or kernel version that no longer exists
> > (filename mismatch). Go into the edit mode, remove the version in
> > front of initrd and vmlinuz and try to use auto-completion (TAB-TAB)
> > to complete the filename and boot again.
> >
> > 3) If nothing helps,  I would have tried to boot it into a single user
> > mode with linux rescue disk and reinstall grub - just to confirm if
> > its grub related or something else.
> >
> > Let me know how it goes,
> >
> > Regards
> > ilya
> >
> >
> >
> > > -----Original Message-----
> > > From: Sean Truman [mailto:struman@gmail.com]
> > > Sent: Monday, May 06, 2013 12:16 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: System VM's hanging
> > >
> > > All,
> > >
> > > My system VM's are hanging at GRUB loading stage2.. using KVM as
> > > hypervisor any idea why this is happening?
> > >
> > > v/r
> > > Sean
> >
> >

Re: System VM's hanging

[Sean Truman <st...@gmail.com>]

I am not seeing any splash screen, I see the Seabios screens and then the
PIX boot line, then GRUB loading stage2.. and the KVM process eats 100% of
one of the cores for about 5 minutes, then the management server tries to
restart.  I am using CS 4.0.2

0) acton-systemvm-02062012.qcow2.bz2
1) I will try this, I ran qemu-img check on all the image files and they
all are fine (I did a virsh dumpxml s-1-vm and looked at the disk it was
referencing), I have also reinstalled SystemVM as per installation
directions.
2)  I cannot even get it to let me get to a point where it will let me edit
grub boot parameters.
3)  Like I said above the System VM doesn't even show me a splash screen.

I have reinstalled management and system templates and am trying it again.
 I will also try to run the system VM's by command line and see if it works
from command line. I had everything working with basic networking, as soon
as I switch it over to advance networking (using VLAN) is when this started
happening.

v/r
Sean


On Mon, May 6, 2013 at 11:36 AM, Musayev, Ilya <im...@webmd.net> wrote:

> Sean,
>
> Specifically what versions of SVM are you running? Do you see a splash
> screen and when you press enter or wait for time out - for this issue to
> occur?
>
> 0) if splash screen is seen and you can make changes to boot linux, remove
> the "quiet" flag from initrd line to see if you can get more meaningful
> errors
>
> 1) I would suggest to change the storage controllers to see if it makes
> any difference.
>
> 2) I've seen in past " GRUB loading stage2" error would occur when
> grub.conf reference initrd or kernel version that no longer exists
> (filename mismatch). Go into the edit mode, remove the version in front of
> initrd and vmlinuz and try to use auto-completion (TAB-TAB) to complete the
> filename and boot again.
>
> 3) If nothing helps,  I would have tried to boot it into a single user
> mode with linux rescue disk and reinstall grub - just to confirm if its
> grub related or something else.
>
> Let me know how it goes,
>
> Regards
> ilya
>
>
>
> > -----Original Message-----
> > From: Sean Truman [mailto:struman@gmail.com]
> > Sent: Monday, May 06, 2013 12:16 PM
> > To: dev@cloudstack.apache.org
> > Subject: System VM's hanging
> >
> > All,
> >
> > My system VM's are hanging at GRUB loading stage2.. using KVM as
> > hypervisor any idea why this is happening?
> >
> > v/r
> > Sean
>
>

RE: System VM's hanging

["Musayev, Ilya" <im...@webmd.net>]

Sean,

Specifically what versions of SVM are you running? Do you see a splash screen and when you press enter or wait for time out - for this issue to occur?

0) if splash screen is seen and you can make changes to boot linux, remove the "quiet" flag from initrd line to see if you can get more meaningful errors

1) I would suggest to change the storage controllers to see if it makes any difference.

2) I've seen in past " GRUB loading stage2" error would occur when grub.conf reference initrd or kernel version that no longer exists (filename mismatch). Go into the edit mode, remove the version in front of initrd and vmlinuz and try to use auto-completion (TAB-TAB) to complete the filename and boot again.

3) If nothing helps,  I would have tried to boot it into a single user mode with linux rescue disk and reinstall grub - just to confirm if its grub related or something else.

Let me know how it goes,

Regards
ilya



> -----Original Message-----
> From: Sean Truman [mailto:struman@gmail.com]
> Sent: Monday, May 06, 2013 12:16 PM
> To: dev@cloudstack.apache.org
> Subject: System VM's hanging
> 
> All,
> 
> My system VM's are hanging at GRUB loading stage2.. using KVM as
> hypervisor any idea why this is happening?
> 
> v/r
> Sean