[SOLVED] Re: SSVM, CPVM not starting after Xen host reboot

[Daniel Hertanu <da...@unixmob.com>]

I was able to fix it eventually:

- click on force reconnect
- immediately click on enter maintenance mode (in less than a minute the
host went to the maintenance mode)
- forgot the NFS SR on the host (not sure if actually required)
- destroyed the bond between NIC1 and NIC2 - that was the bond labeled
"guest"
- deleted the network called cloud_link_local_network
- recreate the bond
- manually added NFS SR and label it according to the name use by
CloudStack (I found it in the logs)
- added the host in CS
- SSVM and CPVM started fine and everything is back to normal


Daniel


On Wed, Oct 30, 2013 at 10:12 PM, Daniel Hertanu <da...@unixmob.com> wrote:

> I tried already a force reconnect with no effect. I would remove
> completely the host and add it back but last time when I've done this the
> host couldn't mount the primary storage anymore, so I would prefer to hear
> some other opinions before doing it. Thanks,
>
> Daniel
>
>
> On Wed, Oct 30, 2013 at 9:50 PM, Ahmad Emneina <ae...@gmail.com> wrote:
>
>> Looks like the desired network doesnt exist on the xen host. I wonder if
>> a force reconnect would fix that? if not lets see if someone more
>> knowledgable can chime in.
>>
>> +Anthony and Kirk, my goto xen people.
>>
>>
>> On Wed, Oct 30, 2013 at 12:40 PM, Daniel Hertanu <da...@unixmob.com>wrote:
>>
>>> Here it is: http://pastebin.com/Av2Rmhf7
>>>
>>>
>>>
>>> On Wed, Oct 30, 2013 at 9:23 PM, Ahmad Emneina <ae...@gmail.com>wrote:
>>>
>>>> got it, so it keeps failing. can you post full logs. it seems XS cant
>>>> bridge the guest nic with the appropriate network for some reason.
>>>>
>>>>
>>>> On Wed, Oct 30, 2013 at 12:04 PM, Daniel Hertanu <da...@unixmob.com>wrote:
>>>>
>>>>> CS is doing it - SSVM number is 307 and CPVM number is 255.
>>>>>
>>>>>
>>>>> On Wed, Oct 30, 2013 at 8:57 PM, Ahmad Emneina <ae...@gmail.com>wrote:
>>>>>
>>>>>> have you tried destroying the ssvm/cpvm? CS should recycle the vm and
>>>>>> spawn
>>>>>> new instances.
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 30, 2013 at 11:03 AM, Daniel Hertanu <da...@unixmob.com>
>>>>>> wrote:
>>>>>>
>>>>>> > One more thing. If I try now to put the host in Maintenance mode,
>>>>>> it stays
>>>>>> > in PrepareforMaintenance. If I cancel the Maintenance mode it
>>>>>> becomes again
>>>>>> > Enabled.
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Wed, Oct 30, 2013 at 8:00 PM, Daniel Hertanu <da...@unixmob.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > > Hi
>>>>>> > >
>>>>>> > > CS 4.2.0 running on CentOS 6.4 64 bit with one XenServer host. I
>>>>>> had to
>>>>>> > > reboot the host so I put the host in maintenance mode. All VMs
>>>>>> were
>>>>>> > > shutdown, then the host was put in Maintenace, then I rebooted
>>>>>> the host.
>>>>>> > > After the reboot both SSVM and CPVM fail to start with something
>>>>>> like
>>>>>> > this:
>>>>>> > >
>>>>>> > >            nameLabel: Async.VM.start_on
>>>>>> > >      nameDescription:
>>>>>> > >    allowedOperations: []
>>>>>> > >    currentOperations: {}
>>>>>> > >              created: Wed Oct 30 17:55:26 GMT 2013
>>>>>> > >             finished: Wed Oct 30 17:55:35 GMT 2013
>>>>>> > >               status: failure
>>>>>> > >           residentOn: com.xensource.xenapi.Host@59ca3c2e
>>>>>> > >             progress: 1.0
>>>>>> > >                 type: <none/>
>>>>>> > >               result:
>>>>>> > >            errorInfo: [CANNOT_PLUG_VIF,
>>>>>> > > OpaqueRef:ce19a694-e572-7620-b1bd-c5a5133f6861]
>>>>>> > >          otherConfig: {}
>>>>>> > >            subtaskOf: com.xensource.xenapi.Task@aaf13f6f
>>>>>> > >             subtasks: []
>>>>>> > >
>>>>>> > >
>>>>>> > > Peeking with XenCenter on the host events log I can see this:
>>>>>> "This VM
>>>>>> > > cannot be started, as its network interfaces could not be
>>>>>> connected. One
>>>>>> > of
>>>>>> > > the NICs is in use elsewhere."
>>>>>> > >
>>>>>> > > In the networking tab the host shows a NIC0 labeled management, a
>>>>>> bond
>>>>>> > > between NIC1 and NIC2 labeled guest, NIC3 labeled storage and a
>>>>>> > > cloud_link_local_network.
>>>>>> > >
>>>>>> > >
>>>>>> > > Any idea on what's going on?
>>>>>> > >
>>>>>> > > Thanks,
>>>>>> > >
>>>>>> > > Daniel
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>