You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Keerthiraja SJ <sj...@gmail.com> on 2015/01/22 07:16:43 UTC
Multiple ROOT disk for one Instances in CS 4.2.1
Hi
For one of my Instances I could see many multiple of RootDisk.
Because of this when xenserver reboots or if I stop and start the instances
my VM is not responding.
How can I fix this issue.
I could see this issue almost out 173 VM's 5 are like this.
*Below are the one pasted from the Storage CS GUI*
ROOT-868ROOTXenServer
ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
ROOT-868ROOTXenServer
ROOT-868ROOTXenServer
ROOT-868ROOTXenServer
ROOT-868ROOTXenServeror004-test.net
ROOT-868ROOTXenServer
ROOT-868ROOTXenServer
ROOT-868ROOTXenServer
ROOT-868ROOTXenServer
Thanks,
Keerthi
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Prashant Kumar Mishra <pr...@citrix.com>.
To investigate further , can you pull out job related to this volume(one
which in expunging state ) from async job table ,and also check storage
cleanup thread in log ,why it was not able to clean this volume.
~prashant
On 1/25/15, 3:04 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>On Sun, Jan 25, 2015 at 6:33 AM, Keerthiraja SJ <sj...@gmail.com>
>wrote:
>> its my production environment. I am facing critical issue when if any
>> xenserver reboots or if I try to patch up the xenserver VM's will not
>> start. In that case I have to create new instances and do all the setup
>> from the scratch, So the downtime will be more than 5 hours.
>>
>> Is there a way we can resolve this issue.
>
>I would guess the following would help:
>
>UPDATE volumes set state = 'Expunged', removed = '2015-01-08
>20:40:00' WHERE instance_id='1381' AND state = 'Expunging';
>
>Note that somehow some call to the backend didn't return and some
>volume dbeb4cf4-0673-48be-b474-ebc5a73d08c7 might still exist
>somewhere (in some primary storage). give it some rest to settle after
>issuing the statement and then check if it helpen by rebooting the vm.
>
>>
>> Thanks,
>> Keerthi
>>
>> On Sat, Jan 24, 2015 at 6:09 PM, Daan Hoogland <da...@gmail.com>
>> wrote:
>>
>>> see inline
>>>
>>> On Sat, Jan 24, 2015 at 6:00 AM, Keerthiraja SJ <sj...@gmail.com>
>>> wrote:
>>> > Below are the query ran for the instances which have many root disk.
>>> >
>>> > mysql> select * from volumes where instance_id='1381';
>>> >
>>>
>>>+------+------------+-----------+---------+--------------+-------------+
>>>-----------+-----------+--------------------------------------+---------
>>>----+-------------------------+--------------------------------------+--
>>>------+----------------+------------+---------+-------------+-----------
>>>+------------------+-------------+----------------------------+---------
>>>----+---------------------+----------+---------------------+---------+--
>>>---------+------------+--------------+-----------+----------------------
>>>--+--------+----------------+--------+----------+----------+
>>> > | id | account_id | domain_id | pool_id | last_pool_id |
>>>instance_id |
>>> > device_id | name | uuid | size
>>> > | folder | path |
>>> pod_id |
>>> > data_center_id | iscsi_name | host_ip | volume_type | pool_type |
>>> > disk_offering_id | template_id | first_snapshot_backup_uuid |
>>> recreatable |
>>> > created | attached | updated | removed |
>>>state
>>> > | chain_info | update_count | disk_type | vm_snapshot_chain_size |
>>> iso_id |
>>> > display_volume | format | min_iops | max_iops |
>>> >
>>>
>>>+------+------------+-----------+---------+--------------+-------------+
>>>-----------+-----------+--------------------------------------+---------
>>>----+-------------------------+--------------------------------------+--
>>>------+----------------+------------+---------+-------------+-----------
>>>+------------------+-------------+----------------------------+---------
>>>----+---------------------+----------+---------------------+---------+--
>>>---------+------------+--------------+-----------+----------------------
>>>--+--------+----------------+--------+----------+----------+
>>> > | 1572 | 2 | 1 | 249 | NULL |
>>>1381
>>> > | 0 | ROOT-1381 | 9ee67199-7af8-4a47-8886-016cfff26b5c |
>>> > 21474836480 | NULL |
>>> > 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | NULL | 5 | NULL
>>> > | NULL | ROOT | NULL | 125 | 347
>>>|
>>> > NULL | 0 | 2015-01-08 14:19:27 | NULL
>>> |
>>> > 2015-01-08 20:39:59 | NULL | Ready | NULL |
>>>4 |
>>> > NULL | NULL | NULL | 0 | VHD
>>> |
>>> > NULL | NULL |
>>> > | 1575 | 2 | 1 | 259 | 249 |
>>>1381
>>> > | 0 | ROOT-1381 | dbeb4cf4-0673-48be-b474-ebc5a73d08c7 |
>>> > 21474836480 | /vol/cldswp_orprimary03 |
>>> > 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | 5 | 5 | NULL
>>> > | NULL | ROOT | NULL | 125 | 347
>>>|
>>> > NULL | 0 | 2015-01-08 20:25:29 | NULL
>>> |
>>> > 2015-01-08 20:39:59 | NULL | Expunging | NULL |
>>>6 |
>>>
>>> here is the problem; the 'Expunging' value indicates that it is in
>>> transit from marked for deletion to actual deletion. I am not sure
>>> what caused this but it is not good. The field should contain the
>>> value 'Expunged' and the field before should contain the time-stamp of
>>> the moment this happened.
>>>
>>> If this is production, investigate before following my advice blindly.
>>> It is code analysis that brought me to this conclusion, not
>>> experience.
>>>
>>> > NULL | NULL | NULL | 0 | VHD
>>> |
>>> > NULL | NULL |
>>> >
>>>
>>>+------+------------+-----------+---------+--------------+-------------+
>>>-----------+-----------+--------------------------------------+---------
>>>----+-------------------------+--------------------------------------+--
>>>------+----------------+------------+---------+-------------+-----------
>>>+------------------+-------------+----------------------------+---------
>>>----+---------------------+----------+---------------------+---------+--
>>>---------+------------+--------------+-----------+----------------------
>>>--+--------+----------------+--------+----------+----------+
>>> > 2 rows in set (0.00 sec)
>>> >
>>> >
>>> >
>>> >
>>> > On Fri, Jan 23, 2015 at 5:00 PM, Prashant Kumar Mishra <
>>> > prashantkumar.mishra@citrix.com> wrote:
>>> >
>>> >> Can you check state of these Disks in DB( volumes table for give
>>> >> instance_id).
>>> >>
>>> >> On 1/23/15, 4:46 PM, "Daan Hoogland" <da...@gmail.com>
>>>wrote:
>>> >>
>>> >> >Keerthi,
>>> >> >
>>> >> >This doesn't ring a bell directly but to investigate further;
>>> >> >
>>> >> >Did you check in the database whether these are pointing to the
>>>same
>>> >> >image? This seems like a data corruption, to me.
>>> >> >
>>> >> >When you say stop/start the vm, do you mean from
>>>cloudstack/xenserver
>>> >> >or within the vm?
>>> >> >
>>> >> >On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ
>>><sj...@gmail.com>
>>> >> >wrote:
>>> >> >> Anyone can help on this.
>>> >> >>
>>> >> >> Thanks,
>>> >> >> Keerthi
>>> >> >>
>>> >> >> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <
>>> sjkeerthi@gmail.com>
>>> >> >> wrote:
>>> >> >>
>>> >> >>> Hi
>>> >> >>>
>>> >> >>> For one of my Instances I could see many multiple of RootDisk.
>>> >> >>>
>>> >> >>> Because of this when xenserver reboots or if I stop and start
>>>the
>>> >> >>> instances my VM is not responding.
>>> >> >>>
>>> >> >>> How can I fix this issue.
>>> >> >>>
>>> >> >>> I could see this issue almost out 173 VM's 5 are like this.
>>> >> >>>
>>> >> >>> *Below are the one pasted from the Storage CS GUI*
>>> >> >>>
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServeror004-test.net
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> ROOT-868ROOTXenServer
>>> >> >>> Thanks,
>>> >> >>> Keerthi
>>> >> >>>
>>> >> >>>
>>> >> >
>>> >> >
>>> >> >
>>> >> >--
>>> >> >Daan
>>> >>
>>> >>
>>>
>>>
>>>
>>> --
>>> Daan
>>>
>
>
>
>--
>Daan
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Daan Hoogland <da...@gmail.com>.
On Sun, Jan 25, 2015 at 6:33 AM, Keerthiraja SJ <sj...@gmail.com> wrote:
> its my production environment. I am facing critical issue when if any
> xenserver reboots or if I try to patch up the xenserver VM's will not
> start. In that case I have to create new instances and do all the setup
> from the scratch, So the downtime will be more than 5 hours.
>
> Is there a way we can resolve this issue.
I would guess the following would help:
UPDATE volumes set state = 'Expunged', removed = '2015-01-08
20:40:00' WHERE instance_id='1381' AND state = 'Expunging';
Note that somehow some call to the backend didn't return and some
volume dbeb4cf4-0673-48be-b474-ebc5a73d08c7 might still exist
somewhere (in some primary storage). give it some rest to settle after
issuing the statement and then check if it helpen by rebooting the vm.
>
> Thanks,
> Keerthi
>
> On Sat, Jan 24, 2015 at 6:09 PM, Daan Hoogland <da...@gmail.com>
> wrote:
>
>> see inline
>>
>> On Sat, Jan 24, 2015 at 6:00 AM, Keerthiraja SJ <sj...@gmail.com>
>> wrote:
>> > Below are the query ran for the instances which have many root disk.
>> >
>> > mysql> select * from volumes where instance_id='1381';
>> >
>> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
>> > | id | account_id | domain_id | pool_id | last_pool_id | instance_id |
>> > device_id | name | uuid | size
>> > | folder | path |
>> pod_id |
>> > data_center_id | iscsi_name | host_ip | volume_type | pool_type |
>> > disk_offering_id | template_id | first_snapshot_backup_uuid |
>> recreatable |
>> > created | attached | updated | removed | state
>> > | chain_info | update_count | disk_type | vm_snapshot_chain_size |
>> iso_id |
>> > display_volume | format | min_iops | max_iops |
>> >
>> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
>> > | 1572 | 2 | 1 | 249 | NULL | 1381
>> > | 0 | ROOT-1381 | 9ee67199-7af8-4a47-8886-016cfff26b5c |
>> > 21474836480 | NULL |
>> > 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | NULL | 5 | NULL
>> > | NULL | ROOT | NULL | 125 | 347 |
>> > NULL | 0 | 2015-01-08 14:19:27 | NULL
>> |
>> > 2015-01-08 20:39:59 | NULL | Ready | NULL | 4 |
>> > NULL | NULL | NULL | 0 | VHD |
>> > NULL | NULL |
>> > | 1575 | 2 | 1 | 259 | 249 | 1381
>> > | 0 | ROOT-1381 | dbeb4cf4-0673-48be-b474-ebc5a73d08c7 |
>> > 21474836480 | /vol/cldswp_orprimary03 |
>> > 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | 5 | 5 | NULL
>> > | NULL | ROOT | NULL | 125 | 347 |
>> > NULL | 0 | 2015-01-08 20:25:29 | NULL
>> |
>> > 2015-01-08 20:39:59 | NULL | Expunging | NULL | 6 |
>>
>> here is the problem; the 'Expunging' value indicates that it is in
>> transit from marked for deletion to actual deletion. I am not sure
>> what caused this but it is not good. The field should contain the
>> value 'Expunged' and the field before should contain the time-stamp of
>> the moment this happened.
>>
>> If this is production, investigate before following my advice blindly.
>> It is code analysis that brought me to this conclusion, not
>> experience.
>>
>> > NULL | NULL | NULL | 0 | VHD |
>> > NULL | NULL |
>> >
>> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
>> > 2 rows in set (0.00 sec)
>> >
>> >
>> >
>> >
>> > On Fri, Jan 23, 2015 at 5:00 PM, Prashant Kumar Mishra <
>> > prashantkumar.mishra@citrix.com> wrote:
>> >
>> >> Can you check state of these Disks in DB( volumes table for give
>> >> instance_id).
>> >>
>> >> On 1/23/15, 4:46 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>> >>
>> >> >Keerthi,
>> >> >
>> >> >This doesn't ring a bell directly but to investigate further;
>> >> >
>> >> >Did you check in the database whether these are pointing to the same
>> >> >image? This seems like a data corruption, to me.
>> >> >
>> >> >When you say stop/start the vm, do you mean from cloudstack/xenserver
>> >> >or within the vm?
>> >> >
>> >> >On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ <sj...@gmail.com>
>> >> >wrote:
>> >> >> Anyone can help on this.
>> >> >>
>> >> >> Thanks,
>> >> >> Keerthi
>> >> >>
>> >> >> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <
>> sjkeerthi@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> Hi
>> >> >>>
>> >> >>> For one of my Instances I could see many multiple of RootDisk.
>> >> >>>
>> >> >>> Because of this when xenserver reboots or if I stop and start the
>> >> >>> instances my VM is not responding.
>> >> >>>
>> >> >>> How can I fix this issue.
>> >> >>>
>> >> >>> I could see this issue almost out 173 VM's 5 are like this.
>> >> >>>
>> >> >>> *Below are the one pasted from the Storage CS GUI*
>> >> >>>
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServeror004-test.net
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> ROOT-868ROOTXenServer
>> >> >>> Thanks,
>> >> >>> Keerthi
>> >> >>>
>> >> >>>
>> >> >
>> >> >
>> >> >
>> >> >--
>> >> >Daan
>> >>
>> >>
>>
>>
>>
>> --
>> Daan
>>
--
Daan
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Keerthiraja SJ <sj...@gmail.com>.
its my production environment. I am facing critical issue when if any
xenserver reboots or if I try to patch up the xenserver VM's will not
start. In that case I have to create new instances and do all the setup
from the scratch, So the downtime will be more than 5 hours.
Is there a way we can resolve this issue.
Thanks,
Keerthi
On Sat, Jan 24, 2015 at 6:09 PM, Daan Hoogland <da...@gmail.com>
wrote:
> see inline
>
> On Sat, Jan 24, 2015 at 6:00 AM, Keerthiraja SJ <sj...@gmail.com>
> wrote:
> > Below are the query ran for the instances which have many root disk.
> >
> > mysql> select * from volumes where instance_id='1381';
> >
> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> > | id | account_id | domain_id | pool_id | last_pool_id | instance_id |
> > device_id | name | uuid | size
> > | folder | path |
> pod_id |
> > data_center_id | iscsi_name | host_ip | volume_type | pool_type |
> > disk_offering_id | template_id | first_snapshot_backup_uuid |
> recreatable |
> > created | attached | updated | removed | state
> > | chain_info | update_count | disk_type | vm_snapshot_chain_size |
> iso_id |
> > display_volume | format | min_iops | max_iops |
> >
> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> > | 1572 | 2 | 1 | 249 | NULL | 1381
> > | 0 | ROOT-1381 | 9ee67199-7af8-4a47-8886-016cfff26b5c |
> > 21474836480 | NULL |
> > 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | NULL | 5 | NULL
> > | NULL | ROOT | NULL | 125 | 347 |
> > NULL | 0 | 2015-01-08 14:19:27 | NULL
> |
> > 2015-01-08 20:39:59 | NULL | Ready | NULL | 4 |
> > NULL | NULL | NULL | 0 | VHD |
> > NULL | NULL |
> > | 1575 | 2 | 1 | 259 | 249 | 1381
> > | 0 | ROOT-1381 | dbeb4cf4-0673-48be-b474-ebc5a73d08c7 |
> > 21474836480 | /vol/cldswp_orprimary03 |
> > 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | 5 | 5 | NULL
> > | NULL | ROOT | NULL | 125 | 347 |
> > NULL | 0 | 2015-01-08 20:25:29 | NULL
> |
> > 2015-01-08 20:39:59 | NULL | Expunging | NULL | 6 |
>
> here is the problem; the 'Expunging' value indicates that it is in
> transit from marked for deletion to actual deletion. I am not sure
> what caused this but it is not good. The field should contain the
> value 'Expunged' and the field before should contain the time-stamp of
> the moment this happened.
>
> If this is production, investigate before following my advice blindly.
> It is code analysis that brought me to this conclusion, not
> experience.
>
> > NULL | NULL | NULL | 0 | VHD |
> > NULL | NULL |
> >
> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> > 2 rows in set (0.00 sec)
> >
> >
> >
> >
> > On Fri, Jan 23, 2015 at 5:00 PM, Prashant Kumar Mishra <
> > prashantkumar.mishra@citrix.com> wrote:
> >
> >> Can you check state of these Disks in DB( volumes table for give
> >> instance_id).
> >>
> >> On 1/23/15, 4:46 PM, "Daan Hoogland" <da...@gmail.com> wrote:
> >>
> >> >Keerthi,
> >> >
> >> >This doesn't ring a bell directly but to investigate further;
> >> >
> >> >Did you check in the database whether these are pointing to the same
> >> >image? This seems like a data corruption, to me.
> >> >
> >> >When you say stop/start the vm, do you mean from cloudstack/xenserver
> >> >or within the vm?
> >> >
> >> >On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ <sj...@gmail.com>
> >> >wrote:
> >> >> Anyone can help on this.
> >> >>
> >> >> Thanks,
> >> >> Keerthi
> >> >>
> >> >> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <
> sjkeerthi@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi
> >> >>>
> >> >>> For one of my Instances I could see many multiple of RootDisk.
> >> >>>
> >> >>> Because of this when xenserver reboots or if I stop and start the
> >> >>> instances my VM is not responding.
> >> >>>
> >> >>> How can I fix this issue.
> >> >>>
> >> >>> I could see this issue almost out 173 VM's 5 are like this.
> >> >>>
> >> >>> *Below are the one pasted from the Storage CS GUI*
> >> >>>
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServeror004-test.net
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServer
> >> >>> ROOT-868ROOTXenServer
> >> >>> Thanks,
> >> >>> Keerthi
> >> >>>
> >> >>>
> >> >
> >> >
> >> >
> >> >--
> >> >Daan
> >>
> >>
>
>
>
> --
> Daan
>
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Daan Hoogland <da...@gmail.com>.
see inline
On Sat, Jan 24, 2015 at 6:00 AM, Keerthiraja SJ <sj...@gmail.com> wrote:
> Below are the query ran for the instances which have many root disk.
>
> mysql> select * from volumes where instance_id='1381';
> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> | id | account_id | domain_id | pool_id | last_pool_id | instance_id |
> device_id | name | uuid | size
> | folder | path | pod_id |
> data_center_id | iscsi_name | host_ip | volume_type | pool_type |
> disk_offering_id | template_id | first_snapshot_backup_uuid | recreatable |
> created | attached | updated | removed | state
> | chain_info | update_count | disk_type | vm_snapshot_chain_size | iso_id |
> display_volume | format | min_iops | max_iops |
> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> | 1572 | 2 | 1 | 249 | NULL | 1381
> | 0 | ROOT-1381 | 9ee67199-7af8-4a47-8886-016cfff26b5c |
> 21474836480 | NULL |
> 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | NULL | 5 | NULL
> | NULL | ROOT | NULL | 125 | 347 |
> NULL | 0 | 2015-01-08 14:19:27 | NULL |
> 2015-01-08 20:39:59 | NULL | Ready | NULL | 4 |
> NULL | NULL | NULL | 0 | VHD |
> NULL | NULL |
> | 1575 | 2 | 1 | 259 | 249 | 1381
> | 0 | ROOT-1381 | dbeb4cf4-0673-48be-b474-ebc5a73d08c7 |
> 21474836480 | /vol/cldswp_orprimary03 |
> 06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | 5 | 5 | NULL
> | NULL | ROOT | NULL | 125 | 347 |
> NULL | 0 | 2015-01-08 20:25:29 | NULL |
> 2015-01-08 20:39:59 | NULL | Expunging | NULL | 6 |
here is the problem; the 'Expunging' value indicates that it is in
transit from marked for deletion to actual deletion. I am not sure
what caused this but it is not good. The field should contain the
value 'Expunged' and the field before should contain the time-stamp of
the moment this happened.
If this is production, investigate before following my advice blindly.
It is code analysis that brought me to this conclusion, not
experience.
> NULL | NULL | NULL | 0 | VHD |
> NULL | NULL |
> +------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> 2 rows in set (0.00 sec)
>
>
>
>
> On Fri, Jan 23, 2015 at 5:00 PM, Prashant Kumar Mishra <
> prashantkumar.mishra@citrix.com> wrote:
>
>> Can you check state of these Disks in DB( volumes table for give
>> instance_id).
>>
>> On 1/23/15, 4:46 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>>
>> >Keerthi,
>> >
>> >This doesn't ring a bell directly but to investigate further;
>> >
>> >Did you check in the database whether these are pointing to the same
>> >image? This seems like a data corruption, to me.
>> >
>> >When you say stop/start the vm, do you mean from cloudstack/xenserver
>> >or within the vm?
>> >
>> >On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ <sj...@gmail.com>
>> >wrote:
>> >> Anyone can help on this.
>> >>
>> >> Thanks,
>> >> Keerthi
>> >>
>> >> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <sj...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi
>> >>>
>> >>> For one of my Instances I could see many multiple of RootDisk.
>> >>>
>> >>> Because of this when xenserver reboots or if I stop and start the
>> >>> instances my VM is not responding.
>> >>>
>> >>> How can I fix this issue.
>> >>>
>> >>> I could see this issue almost out 173 VM's 5 are like this.
>> >>>
>> >>> *Below are the one pasted from the Storage CS GUI*
>> >>>
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServeror004-test.net
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServer
>> >>> ROOT-868ROOTXenServer
>> >>> Thanks,
>> >>> Keerthi
>> >>>
>> >>>
>> >
>> >
>> >
>> >--
>> >Daan
>>
>>
--
Daan
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Keerthiraja SJ <sj...@gmail.com>.
Below are the query ran for the instances which have many root disk.
mysql> select * from volumes where instance_id='1381';
+------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
| id | account_id | domain_id | pool_id | last_pool_id | instance_id |
device_id | name | uuid | size
| folder | path | pod_id |
data_center_id | iscsi_name | host_ip | volume_type | pool_type |
disk_offering_id | template_id | first_snapshot_backup_uuid | recreatable |
created | attached | updated | removed | state
| chain_info | update_count | disk_type | vm_snapshot_chain_size | iso_id |
display_volume | format | min_iops | max_iops |
+------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
| 1572 | 2 | 1 | 249 | NULL | 1381
| 0 | ROOT-1381 | 9ee67199-7af8-4a47-8886-016cfff26b5c |
21474836480 | NULL |
06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | NULL | 5 | NULL
| NULL | ROOT | NULL | 125 | 347 |
NULL | 0 | 2015-01-08 14:19:27 | NULL |
2015-01-08 20:39:59 | NULL | Ready | NULL | 4 |
NULL | NULL | NULL | 0 | VHD |
NULL | NULL |
| 1575 | 2 | 1 | 259 | 249 | 1381
| 0 | ROOT-1381 | dbeb4cf4-0673-48be-b474-ebc5a73d08c7 |
21474836480 | /vol/cldswp_orprimary03 |
06a3b4c9-6cfe-45c8-8640-7f562b11e2cb | 5 | 5 | NULL
| NULL | ROOT | NULL | 125 | 347 |
NULL | 0 | 2015-01-08 20:25:29 | NULL |
2015-01-08 20:39:59 | NULL | Expunging | NULL | 6 |
NULL | NULL | NULL | 0 | VHD |
NULL | NULL |
+------+------------+-----------+---------+--------------+-------------+-----------+-----------+--------------------------------------+-------------+-------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
2 rows in set (0.00 sec)
On Fri, Jan 23, 2015 at 5:00 PM, Prashant Kumar Mishra <
prashantkumar.mishra@citrix.com> wrote:
> Can you check state of these Disks in DB( volumes table for give
> instance_id).
>
> On 1/23/15, 4:46 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>
> >Keerthi,
> >
> >This doesn't ring a bell directly but to investigate further;
> >
> >Did you check in the database whether these are pointing to the same
> >image? This seems like a data corruption, to me.
> >
> >When you say stop/start the vm, do you mean from cloudstack/xenserver
> >or within the vm?
> >
> >On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ <sj...@gmail.com>
> >wrote:
> >> Anyone can help on this.
> >>
> >> Thanks,
> >> Keerthi
> >>
> >> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <sj...@gmail.com>
> >> wrote:
> >>
> >>> Hi
> >>>
> >>> For one of my Instances I could see many multiple of RootDisk.
> >>>
> >>> Because of this when xenserver reboots or if I stop and start the
> >>> instances my VM is not responding.
> >>>
> >>> How can I fix this issue.
> >>>
> >>> I could see this issue almost out 173 VM's 5 are like this.
> >>>
> >>> *Below are the one pasted from the Storage CS GUI*
> >>>
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServeror004-test.net
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServer
> >>> ROOT-868ROOTXenServer
> >>> Thanks,
> >>> Keerthi
> >>>
> >>>
> >
> >
> >
> >--
> >Daan
>
>
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Prashant Kumar Mishra <pr...@citrix.com>.
Can you check state of these Disks in DB( volumes table for give
instance_id).
On 1/23/15, 4:46 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>Keerthi,
>
>This doesn't ring a bell directly but to investigate further;
>
>Did you check in the database whether these are pointing to the same
>image? This seems like a data corruption, to me.
>
>When you say stop/start the vm, do you mean from cloudstack/xenserver
>or within the vm?
>
>On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ <sj...@gmail.com>
>wrote:
>> Anyone can help on this.
>>
>> Thanks,
>> Keerthi
>>
>> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <sj...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> For one of my Instances I could see many multiple of RootDisk.
>>>
>>> Because of this when xenserver reboots or if I stop and start the
>>> instances my VM is not responding.
>>>
>>> How can I fix this issue.
>>>
>>> I could see this issue almost out 173 VM's 5 are like this.
>>>
>>> *Below are the one pasted from the Storage CS GUI*
>>>
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServeror004-test.net
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServer
>>> ROOT-868ROOTXenServer
>>> Thanks,
>>> Keerthi
>>>
>>>
>
>
>
>--
>Daan
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Daan Hoogland <da...@gmail.com>.
Keerthi,
This doesn't ring a bell directly but to investigate further;
Did you check in the database whether these are pointing to the same
image? This seems like a data corruption, to me.
When you say stop/start the vm, do you mean from cloudstack/xenserver
or within the vm?
On Fri, Jan 23, 2015 at 5:47 AM, Keerthiraja SJ <sj...@gmail.com> wrote:
> Anyone can help on this.
>
> Thanks,
> Keerthi
>
> On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <sj...@gmail.com>
> wrote:
>
>> Hi
>>
>> For one of my Instances I could see many multiple of RootDisk.
>>
>> Because of this when xenserver reboots or if I stop and start the
>> instances my VM is not responding.
>>
>> How can I fix this issue.
>>
>> I could see this issue almost out 173 VM's 5 are like this.
>>
>> *Below are the one pasted from the Storage CS GUI*
>>
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServeror004-test.net
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServer
>> ROOT-868ROOTXenServer
>> Thanks,
>> Keerthi
>>
>>
--
Daan
Re: Multiple ROOT disk for one Instances in CS 4.2.1
Posted by Keerthiraja SJ <sj...@gmail.com>.
Anyone can help on this.
Thanks,
Keerthi
On Thu, Jan 22, 2015 at 11:46 AM, Keerthiraja SJ <sj...@gmail.com>
wrote:
> Hi
>
> For one of my Instances I could see many multiple of RootDisk.
>
> Because of this when xenserver reboots or if I stop and start the
> instances my VM is not responding.
>
> How can I fix this issue.
>
> I could see this issue almost out 173 VM's 5 are like this.
>
> *Below are the one pasted from the Storage CS GUI*
>
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServeror004-test.net ROOT-868ROOTXenServer
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServeror004-test.net
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServer
> ROOT-868ROOTXenServer
> Thanks,
> Keerthi
>
>