You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Lu Heng <h....@anytimechinese.com> on 2012/06/12 04:29:08 UTC
Secondary storage is not working
Hi
I have secondary storage setup at management server's /data path, and
while I add it to the zone, it is not working(with information
server:<management-server-IP> path:/data), however,I can successfully mount
secondary storage in the HV server, any one have idea what is wrong?
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
RE: Secondary storage is not working
Posted by Nitin Mehta <Ni...@citrix.com>.
Lu - Have you seeded your secondary storage with system vm templates ? That seems to be the issue here.
-----Original Message-----
From: Lu Heng [mailto:h.lu@anytimechinese.com]
Sent: Tuesday, June 12, 2012 8:43 AM
To: cloudstack-dev@incubator.apache.org
Subject: Re: Secondary storage is not working
following data is the error log:(ip has been replaced, rest is same)
2012-06-11 20:00:41,530 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-369:null) Catch Exception
com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs:// 1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,531 DEBUG [agent.transport.Request]
(DirectAgent-369:null) Seq 6-1550453578: Processing: { Ans: , MgmtId:
229144658033003, via: 6, Ver: v1, Flags: 110, [{"storage.PrimaryStorageDownloadAnswer":{"templateSize":0,"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs:// 1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde","wait":0}}] }
2012-06-11 20:00:41,531 WARN [cloud.vm.VirtualMachineManagerImpl]
(DirectAgent-369:null) Cleanup failed due to Catch Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs:// 1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,531 WARN [cloud.vm.VirtualMachineManagerImpl]
(consoleproxy-1:null) Cleanup failed due to Catch Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs:// 1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,532 DEBUG [agent.manager.AgentManagerImpl]
(consoleproxy-1:null) Details from executing class
com.cloud.agent.api.storage.PrimaryStorageDownloadCommand: Catch Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs:// 1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,532 DEBUG [cloud.template.TemplateManagerImpl]
(consoleproxy-1:null) Template 1 download to pool 210 failed due to Catch Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs:// 1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:43,784 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-189:null) destoryVDIbyNameLabel failed due to there are 0 VDIs with name cloud-36981559-0786-40b4-9eff-c7c4cb862731
2012-06-11 20:00:43,784 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-189:null) can't mount 1.1.128.2:/data/template/tmpl/1/1 to
/var/run/cloud_mount/2fcaba54-8c01-4c0f-b3d7-03eb387b42d2
On Tue, Jun 12, 2012 at 4:29 AM, Lu Heng <h....@anytimechinese.com> wrote:
> Hi
>
> I have secondary storage setup at management server's /data path, and
> while I add it to the zone, it is not working(with information
> server:<management-server-IP> path:/data), however,I can successfully
> mount secondary storage in the HV server, any one have idea what is wrong?
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.
>
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
Re: Secondary storage is not working
Posted by Lu Heng <h....@anytimechinese.com>.
following data is the error log:(ip has been replaced, rest is same)
2012-06-11 20:00:41,530 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-369:null) Catch Exception
com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs://
1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,531 DEBUG [agent.transport.Request]
(DirectAgent-369:null) Seq 6-1550453578: Processing: { Ans: , MgmtId:
229144658033003, via: 6, Ver: v1, Flags: 110,
[{"storage.PrimaryStorageDownloadAnswer":{"templateSize":0,"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs://
1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde","wait":0}}] }
2012-06-11 20:00:41,531 WARN [cloud.vm.VirtualMachineManagerImpl]
(DirectAgent-369:null) Cleanup failed due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs://
1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,531 WARN [cloud.vm.VirtualMachineManagerImpl]
(consoleproxy-1:null) Cleanup failed due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs://
1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,532 DEBUG [agent.manager.AgentManagerImpl]
(consoleproxy-1:null) Details from executing class
com.cloud.agent.api.storage.PrimaryStorageDownloadCommand: Catch Exception
com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs://
1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:41,532 DEBUG [cloud.template.TemplateManagerImpl]
(consoleproxy-1:null) Template 1 download to pool 210 failed due to Catch
Exception com.cloud.utils.exception.CloudRuntimeException on
host:b9f6241d-cbd6-49a8-916e-dff524569dd4 for template: nfs://
1.1.128.2/data/template/tmpl/1/1/ due to
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde
2012-06-11 20:00:43,784 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-189:null) destoryVDIbyNameLabel failed due to there are 0 VDIs
with name cloud-36981559-0786-40b4-9eff-c7c4cb862731
2012-06-11 20:00:43,784 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-189:null) can't mount 1.1.128.2:/data/template/tmpl/1/1 to
/var/run/cloud_mount/2fcaba54-8c01-4c0f-b3d7-03eb387b42d2
On Tue, Jun 12, 2012 at 4:29 AM, Lu Heng <h....@anytimechinese.com> wrote:
> Hi
>
> I have secondary storage setup at management server's /data path, and
> while I add it to the zone, it is not working(with information
> server:<management-server-IP> path:/data), however,I can successfully mount
> secondary storage in the HV server, any one have idea what is wrong?
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.
>
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
Re: Secondary storage is not working
Posted by Lu Heng <h....@anytimechinese.com>.
Hi
I have following setup, and I have one quick question, how the primary
storage talk to the secondary storage?
1 cp
1 nic on management network
2 hv(hyperviser server)
1 nic on management network
1 nic on storage network
1 nic on guest network
1 storage
1 nic on storage network
1 nic on management network.
As you can see, the primary storage is added on storage network, in which
it does not talk to the management network, and the NFS is on the
management server as it only have access to the management network.
We have this setup was because we used onapp, and we switch from onapp
because they have very poor support team(as I have experiencing not even as
good as the free community support I got here). With onapp, this setup, the
HV read secondary storage from management network, and read primary storage
from the storage network.
So the question comes, if we use management server as shared NFS, how can
this NFS talk to the primary storage? though HV server, or though storage
network?
If it talk though storage network, then we know where the problem is.
On Tue, Jun 12, 2012 at 3:21 PM, Nitin Mehta <Ni...@citrix.com> wrote:
> Please try this and I can look into your setup tomorrow****
>
> ** **
>
> *From:* Nitin Mehta
> *Sent:* Tuesday, June 12, 2012 6:51 PM
> *To:* 'Lu Heng'
> *Subject:* RE: Secondary storage is not working****
>
> ** **
>
> There is a file by the name of copy_vhd_from_secondarystorage.sh which
> tries to copy template from secondary to primary and that is what is
> failing.****
>
> cd /opt/xensource/bin/ and you will find that.****
>
> If you notice it was failing at the following point in the code. See if
> you can manually run this command (also see that there is no mount already
> existing in the xenserver )****
>
> ** **
>
> mount -o *tcp*,soft,*ro*,*timeo*=133,*retrans*=1 $*mountpoint* $*localmp**
> ***
>
> if [ $? -*ne* 0 ]; then****
>
> echo "6#can't mount $*mountpoint* to $*localmp*"****
>
> exit 0****
>
> *fi*
>
> * *
>
> If all of this is working what I suggest is cleaning up the xenserver
> (best is reinstalling it) and then redo the entire setup (remember to turn
> the verbose logs by removing the comment (‘#’) from set –x in the script).
> ****
>
> ** **
>
> *From:* Lu Heng [mailto:h.lu@anytimechinese.com]
> *Sent:* Tuesday, June 12, 2012 5:47 PM
> *To:* Nitin Mehta
>
> *Subject:* Re: Secondary storage is not working****
>
> ** **
>
> Hi Nitin:****
>
> ** **
>
> Just in case if you interested, I can send you login to my cloudstack
> admin panel as well as root login to cp and HV. so that you can see
> yourself.****
>
> On Tue, Jun 12, 2012 at 2:09 PM, Lu Heng <h....@anytimechinese.com> wrote:*
> ***
>
> Hi****
>
> ** **
>
> I didn't see anything interesting here. Here is the most recent
> version(but I have looked at the rotated vew ones, looks the same)****
>
> ** **
>
> [13125] 2012-06-12 05:01:41.648419 #### VMOPS enter get_rule_logs_for_vms
> ####****
>
> [13125] 2012-06-12 05:01:41.683571 #### VMOPS enter
> network_rules_for_rebooted_vm ####****
>
> [13125] 2012-06-12 05:01:41.683614 #### VMOPS enter check_domid_changed
> ####****
>
> [13125] 2012-06-12 05:01:41.693652 #### VMOPS exit check_domid_changed
> ####****
>
> [13125] 2012-06-12 05:01:41.693690 #### VMOPS exit
> network_rules_for_rebooted_vm ####****
>
> [13125] 2012-06-12 05:01:41.693732 #### VMOPS exit get_rule_logs_for_vms
> ####****
>
> [13181] 2012-06-12 05:01:49.672472 #### VMOPS enter gethostvmstats ####**
> **
>
> [13181] 2012-06-12 05:01:49.680152 #### VMOPS exit gethostvmstats ####***
> *
>
> [13189] 2012-06-12 05:01:50.688317 Setting LVM_DEVICE to
> /dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439****
>
> [13189] 2012-06-12 05:01:50.696646 Setting LVM_DEVICE to
> /dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439****
>
> [13189] 2012-06-12 05:01:50.734480 lock: acquired
> /var/lock/sm/iscsiadm/running****
>
> [13189] 2012-06-12 05:01:50.746575 lock: released
> /var/lock/sm/iscsiadm/running****
>
> [13189] 2012-06-12 05:01:50.746687 lock: closed
> /var/lock/sm/iscsiadm/running****
>
> [13189] 2012-06-12 05:01:50.747452 ['ls', '/sys/class/scsi_host', '-1',
> '--color=never']****
>
> [13189] 2012-06-12 05:01:50.759330 SUCCESS****
>
> [13189] 2012-06-12 05:01:50.759950 ['11']****
>
> [13189] 2012-06-12 05:01:50.763759 PATHDICT: key 10.2.0.5:3260: {'path':
> '/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
> 'ipaddr': '10.2.0.5', 'port': 3260}****
>
> [13189] 2012-06-12 05:01:50.763857 Path found:
> /dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260****
>
> [13189] 2012-06-12 05:01:50.777597 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13189] 2012-06-12 05:01:50.792979 ['/usr/sbin/vgs',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13189] 2012-06-12 05:01:50.815790 SUCCESS****
>
> [13189] 2012-06-12 05:01:50.816004 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:50.816106 LVMCache: will initialize now****
>
> [13189] 2012-06-12 05:01:50.816175 LVMCache: refreshing****
>
> [13189] 2012-06-12 05:01:50.816269 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13189] 2012-06-12 05:01:50.839279 SUCCESS****
>
> [13189] 2012-06-12 05:01:50.839672 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:50.839784 Entering _checkMetadataVolume****
>
> [13189] 2012-06-12 05:01:50.841193 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:50.841309 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13189] 2012-06-12 05:01:50.856727 ['/usr/sbin/vgs',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13189] 2012-06-12 05:01:50.878072 SUCCESS****
>
> [13189] 2012-06-12 05:01:50.878301 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:50.878406 LVMCache: will initialize now****
>
> [13189] 2012-06-12 05:01:50.878475 LVMCache: refreshing****
>
> [13189] 2012-06-12 05:01:50.878568 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13189] 2012-06-12 05:01:50.900543 SUCCESS****
>
> [13189] 2012-06-12 05:01:50.900936 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:50.901037 Entering _checkMetadataVolume****
>
> [13189] 2012-06-12 05:01:50.901347 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:50.901531 sr_scan {'sr_uuid':
> '900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
> 'OpaqueRef:ebe4d090-bf89-d8d4-c200-591d95395fa5', 'args': [], 'host_ref':
> 'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
> 'OpaqueRef:d89486fe-07fc-182b-0ba9-f770211adfdc', 'device_config':
> {'device':
> '/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
> 'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
> 'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
> '10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
> 'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
> 'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}****
>
> [13189] 2012-06-12 05:01:50.901698 LVHDSR.scan for
> 900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13189] 2012-06-12 05:01:50.901922 ['/usr/sbin/vhd-util', 'scan', '-f',
> '-c', '-m', 'VHD-*', '-l',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13189] 2012-06-12 05:01:50.943280 SUCCESS****
>
> [13189] 2012-06-12 05:01:50.944136 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13189] 2012-06-12 05:01:50.965564 SUCCESS****
>
> [13189] 2012-06-12 05:01:51.038455 lock: tried lock
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
> (exists: True)****
>
> [13189] 2012-06-12 05:01:51.038507 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running****
>
> [13189] 2012-06-12 05:01:51.038537 Kicking GC****
>
> <13189> 2012-06-12 05:01:51.038575 === SR
> 900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===****
>
> <13223> 2012-06-12 05:01:51.040066 Will finish as PID [13224]****
>
> <13189> 2012-06-12 05:01:51.040723 New PID [13223]****
>
> [13189] 2012-06-12 05:01:51.040941 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running****
>
> [13189] 2012-06-12 05:01:51.041280 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13189] 2012-06-12 05:01:51.047912 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13224] 2012-06-12 05:01:51.094792 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13224] 2012-06-12 05:01:51.112970 lock: tried lock
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
> (exists: True)****
>
> [13224] 2012-06-12 05:01:51.113013 LVMCache: refreshing****
>
> [13224] 2012-06-12 05:01:51.113050 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13224] 2012-06-12 05:01:51.136424 SUCCESS****
>
> [13224] 2012-06-12 05:01:51.136869 ['/usr/sbin/vhd-util', 'scan', '-f',
> '-c', '-m', 'VHD-*', '-l',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13224] 2012-06-12 05:01:51.179332 SUCCESS****
>
> <13224> 2012-06-12 05:01:51.180168 SR 900f
> ('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): ****
>
> *0de05e6d[VHD](2.000G//540.000M|ao)****
>
> fedba9ba[VHD](2.000G//2.012G|ao)****
>
> 5bca5a2b[VHD](2.000G//2.012G|n)****
>
> bf63b940[VHD](2.000G//8.000M|n)****
>
> ** **
>
> [13224] 2012-06-12 05:01:51.180354 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13224] 2012-06-12 05:01:51.180523 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13224] 2012-06-12 05:01:51.201612 SUCCESS****
>
> [13224] 2012-06-12 05:01:51.201856 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13224] 2012-06-12 05:01:51.223280 SUCCESS****
>
> <13224> 2012-06-12 05:01:51.238128 Got sm-config for
> *0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
> 'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
> ****
>
> <13224> 2012-06-12 05:01:51.238235 No work, exiting****
>
> <13224> 2012-06-12 05:01:51.238473 SR 900f
> ('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
> ****
>
> [13367] 2012-06-12 05:02:41.632069 #### VMOPS enter get_rule_logs_for_vms
> ####****
>
> [13367] 2012-06-12 05:02:41.657769 #### VMOPS enter
> network_rules_for_rebooted_vm ####****
>
> [13367] 2012-06-12 05:02:41.657863 #### VMOPS enter check_domid_changed
> ####****
>
> [13367] 2012-06-12 05:02:41.669337 #### VMOPS exit check_domid_changed
> ####****
>
> [13367] 2012-06-12 05:02:41.669374 #### VMOPS exit
> network_rules_for_rebooted_vm ####****
>
> [13367] 2012-06-12 05:02:41.669416 #### VMOPS exit get_rule_logs_for_vms
> ####****
>
> [13412] 2012-06-12 05:02:50.192681 #### VMOPS enter gethostvmstats ####**
> **
>
> [13412] 2012-06-12 05:02:50.200365 #### VMOPS exit gethostvmstats ####***
> *
>
> [13420] 2012-06-12 05:02:51.214194 Setting LVM_DEVICE to
> /dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439****
>
> [13420] 2012-06-12 05:02:51.222629 Setting LVM_DEVICE to
> /dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439****
>
> [13420] 2012-06-12 05:02:51.260176 lock: acquired
> /var/lock/sm/iscsiadm/running****
>
> [13420] 2012-06-12 05:02:51.271762 lock: released
> /var/lock/sm/iscsiadm/running****
>
> [13420] 2012-06-12 05:02:51.271872 lock: closed
> /var/lock/sm/iscsiadm/running****
>
> [13420] 2012-06-12 05:02:51.272634 ['ls', '/sys/class/scsi_host', '-1',
> '--color=never']****
>
> [13420] 2012-06-12 05:02:51.284386 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.285007 ['11']****
>
> [13420] 2012-06-12 05:02:51.288837 PATHDICT: key 10.2.0.5:3260: {'path':
> '/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
> 'ipaddr': '10.2.0.5', 'port': 3260}****
>
> [13420] 2012-06-12 05:02:51.288936 Path found:
> /dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260****
>
> [13420] 2012-06-12 05:02:51.302907 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13420] 2012-06-12 05:02:51.318194 ['/usr/sbin/vgs',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13420] 2012-06-12 05:02:51.339945 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.340163 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.340265 LVMCache: will initialize now****
>
> [13420] 2012-06-12 05:02:51.340334 LVMCache: refreshing****
>
> [13420] 2012-06-12 05:02:51.340431 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13420] 2012-06-12 05:02:51.363255 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.363653 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.363758 Entering _checkMetadataVolume****
>
> [13420] 2012-06-12 05:02:51.365197 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.365314 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13420] 2012-06-12 05:02:51.380298 ['/usr/sbin/vgs',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13420] 2012-06-12 05:02:51.401257 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.401471 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.401572 LVMCache: will initialize now****
>
> [13420] 2012-06-12 05:02:51.401641 LVMCache: refreshing****
>
> [13420] 2012-06-12 05:02:51.401734 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13420] 2012-06-12 05:02:51.424356 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.424750 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.424856 Entering _checkMetadataVolume****
>
> [13420] 2012-06-12 05:02:51.425167 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.425353 sr_scan {'sr_uuid':
> '900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
> 'OpaqueRef:672a5cf7-d938-c6bd-001f-7b59d4a69204', 'args': [], 'host_ref':
> 'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
> 'OpaqueRef:af2690fc-6539-daf2-ff58-30fe2f9a0371', 'device_config':
> {'device':
> '/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
> 'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
> 'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
> '10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
> 'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
> 'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}****
>
> [13420] 2012-06-12 05:02:51.425503 LVHDSR.scan for
> 900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13420] 2012-06-12 05:02:51.425726 ['/usr/sbin/vhd-util', 'scan', '-f',
> '-c', '-m', 'VHD-*', '-l',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13420] 2012-06-12 05:02:51.468757 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.469610 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13420] 2012-06-12 05:02:51.491025 SUCCESS****
>
> [13420] 2012-06-12 05:02:51.564940 lock: tried lock
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
> (exists: True)****
>
> [13420] 2012-06-12 05:02:51.564993 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running****
>
> [13420] 2012-06-12 05:02:51.565033 Kicking GC****
>
> <13420> 2012-06-12 05:02:51.565073 === SR
> 900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===****
>
> <13454> 2012-06-12 05:02:51.566603 Will finish as PID [13455]****
>
> <13420> 2012-06-12 05:02:51.567259 New PID [13454]****
>
> [13420] 2012-06-12 05:02:51.567476 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running****
>
> [13420] 2012-06-12 05:02:51.567812 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13420] 2012-06-12 05:02:51.574404 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13455] 2012-06-12 05:02:51.621214 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13455] 2012-06-12 05:02:51.639586 lock: tried lock
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
> (exists: True)****
>
> [13455] 2012-06-12 05:02:51.639629 LVMCache: refreshing****
>
> [13455] 2012-06-12 05:02:51.639667 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13455] 2012-06-12 05:02:51.662316 SUCCESS****
>
> [13455] 2012-06-12 05:02:51.662762 ['/usr/sbin/vhd-util', 'scan', '-f',
> '-c', '-m', 'VHD-*', '-l',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13455] 2012-06-12 05:02:51.704936 SUCCESS****
>
> <13455> 2012-06-12 05:02:51.705771 SR 900f
> ('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): ****
>
> *0de05e6d[VHD](2.000G//540.000M|ao)****
>
> fedba9ba[VHD](2.000G//2.012G|ao)****
>
> 5bca5a2b[VHD](2.000G//2.012G|n)****
>
> bf63b940[VHD](2.000G//8.000M|n)****
>
> ** **
>
> [13455] 2012-06-12 05:02:51.705959 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13455] 2012-06-12 05:02:51.706127 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13455] 2012-06-12 05:02:51.727751 SUCCESS****
>
> [13455] 2012-06-12 05:02:51.727992 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13455] 2012-06-12 05:02:51.748946 SUCCESS****
>
> <13455> 2012-06-12 05:02:51.761594 Got sm-config for
> *0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
> 'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
> ****
>
> <13455> 2012-06-12 05:02:51.761638 No work, exiting****
>
> <13455> 2012-06-12 05:02:51.761737 SR 900f
> ('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
> ****
>
> [13614] 2012-06-12 05:03:41.631406 #### VMOPS enter get_rule_logs_for_vms
> ####****
>
> [13614] 2012-06-12 05:03:41.656623 #### VMOPS enter
> network_rules_for_rebooted_vm ####****
>
> [13614] 2012-06-12 05:03:41.656718 #### VMOPS enter check_domid_changed
> ####****
>
> [13614] 2012-06-12 05:03:41.668513 #### VMOPS exit check_domid_changed
> ####****
>
> [13614] 2012-06-12 05:03:41.668551 #### VMOPS exit
> network_rules_for_rebooted_vm ####****
>
> [13614] 2012-06-12 05:03:41.668593 #### VMOPS exit get_rule_logs_for_vms
> ####****
>
> [13659] 2012-06-12 05:03:50.674948 #### VMOPS enter gethostvmstats ####**
> **
>
> [13659] 2012-06-12 05:03:50.682474 #### VMOPS exit gethostvmstats ####***
> *
>
> [13669] 2012-06-12 05:03:51.816879 Setting LVM_DEVICE to
> /dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439****
>
> [13669] 2012-06-12 05:03:51.825230 Setting LVM_DEVICE to
> /dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439****
>
> [13669] 2012-06-12 05:03:51.861581 lock: acquired
> /var/lock/sm/iscsiadm/running****
>
> [13669] 2012-06-12 05:03:51.873663 lock: released
> /var/lock/sm/iscsiadm/running****
>
> [13669] 2012-06-12 05:03:51.873776 lock: closed
> /var/lock/sm/iscsiadm/running****
>
> [13669] 2012-06-12 05:03:51.874533 ['ls', '/sys/class/scsi_host', '-1',
> '--color=never']****
>
> [13669] 2012-06-12 05:03:51.884299 SUCCESS****
>
> [13669] 2012-06-12 05:03:51.884918 ['11']****
>
> [13669] 2012-06-12 05:03:51.888744 PATHDICT: key 10.2.0.5:3260: {'path':
> '/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
> 'ipaddr': '10.2.0.5', 'port': 3260}****
>
> [13669] 2012-06-12 05:03:51.888843 Path found:
> /dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260****
>
> [13669] 2012-06-12 05:03:51.902943 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13669] 2012-06-12 05:03:51.918180 ['/usr/sbin/vgs',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13669] 2012-06-12 05:03:51.940914 SUCCESS****
>
> [13669] 2012-06-12 05:03:51.941132 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:51.941234 LVMCache: will initialize now****
>
> [13669] 2012-06-12 05:03:51.941304 LVMCache: refreshing****
>
> [13669] 2012-06-12 05:03:51.941401 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13669] 2012-06-12 05:03:51.964523 SUCCESS****
>
> [13669] 2012-06-12 05:03:51.964912 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:51.965019 Entering _checkMetadataVolume****
>
> [13669] 2012-06-12 05:03:51.966443 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:51.966560 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13669] 2012-06-12 05:03:51.982203 ['/usr/sbin/vgs',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13669] 2012-06-12 05:03:52.003569 SUCCESS****
>
> [13669] 2012-06-12 05:03:52.003665 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:52.003711 LVMCache: will initialize now****
>
> [13669] 2012-06-12 05:03:52.003741 LVMCache: refreshing****
>
> [13669] 2012-06-12 05:03:52.003784 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13669] 2012-06-12 05:03:52.025826 SUCCESS****
>
> [13669] 2012-06-12 05:03:52.026232 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:52.026336 Entering _checkMetadataVolume****
>
> [13669] 2012-06-12 05:03:52.026648 lock: acquired
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:52.026834 sr_scan {'sr_uuid':
> '900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
> 'OpaqueRef:70e76ca3-6e80-c865-98bb-9dbfa362379a', 'args': [], 'host_ref':
> 'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
> 'OpaqueRef:2aa1c42e-3b05-c271-bf60-57428a855e46', 'device_config':
> {'device':
> '/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
> 'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
> 'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
> '10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
> 'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
> 'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}****
>
> [13669] 2012-06-12 05:03:52.026984 LVHDSR.scan for
> 900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13669] 2012-06-12 05:03:52.027207 ['/usr/sbin/vhd-util', 'scan', '-f',
> '-c', '-m', 'VHD-*', '-l',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13669] 2012-06-12 05:03:52.070068 SUCCESS****
>
> [13669] 2012-06-12 05:03:52.070933 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13669] 2012-06-12 05:03:52.092699 SUCCESS****
>
> [13669] 2012-06-12 05:03:52.160000 lock: tried lock
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
> (exists: True)****
>
> [13669] 2012-06-12 05:03:52.160052 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running****
>
> [13669] 2012-06-12 05:03:52.160082 Kicking GC****
>
> <13669> 2012-06-12 05:03:52.160121 === SR
> 900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===****
>
> <13703> 2012-06-12 05:03:52.161610 Will finish as PID [13704]****
>
> <13669> 2012-06-12 05:03:52.162260 New PID [13703]****
>
> [13669] 2012-06-12 05:03:52.162484 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running****
>
> [13669] 2012-06-12 05:03:52.162823 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13669] 2012-06-12 05:03:52.169460 lock: closed
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13704] 2012-06-12 05:03:52.221888 LVMCache created for
> VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4****
>
> [13704] 2012-06-12 05:03:52.256927 lock: tried lock
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
> (exists: True)****
>
> [13704] 2012-06-12 05:03:52.257025 LVMCache: refreshing****
>
> [13704] 2012-06-12 05:03:52.257133 ['/usr/sbin/lvs', '--noheadings',
> '--units', 'b', '-o', '+lv_tags',
> '/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13704] 2012-06-12 05:03:52.282878 SUCCESS****
>
> [13704] 2012-06-12 05:03:52.283329 ['/usr/sbin/vhd-util', 'scan', '-f',
> '-c', '-m', 'VHD-*', '-l',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13704] 2012-06-12 05:03:52.332991 SUCCESS****
>
> <13704> 2012-06-12 05:03:52.333830 SR 900f
> ('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): ****
>
> *0de05e6d[VHD](2.000G//540.000M|ao)****
>
> fedba9ba[VHD](2.000G//2.012G|ao)****
>
> 5bca5a2b[VHD](2.000G//2.012G|n)****
>
> bf63b940[VHD](2.000G//8.000M|n)****
>
> ** **
>
> [13704] 2012-06-12 05:03:52.334019 lock: released
> /var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr****
>
> [13704] 2012-06-12 05:03:52.334186 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13704] 2012-06-12 05:03:52.354072 SUCCESS****
>
> [13704] 2012-06-12 05:03:52.354315 ['/usr/sbin/vgs', '--noheadings',
> '--nosuffix', '--units', 'b',
> 'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']****
>
> [13704] 2012-06-12 05:03:52.376106 SUCCESS****
>
> <13704> 2012-06-12 05:03:52.393268 Got sm-config for
> *0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
> 'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
> ****
>
> <13704> 2012-06-12 05:03:52.393377 No work, exiting****
>
> <13704> 2012-06-12 05:03:52.393603 SR 900f
> ('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
> ****
>
> ** **
>
> On Tue, Jun 12, 2012 at 1:21 PM, Nitin Mehta <Ni...@citrix.com>
> wrote:****
>
> I am assuming you are using Xenserver. Log in to the hypervisor. “cd
> /var/log/”. Here you can find the logs. These logs get rotated so be sure
> to search in all of them.****
>
> I am asking to you to look here because the operation
> PrimaryStorageDownloadCommand is failing which downloads the template from
> secondary to primary storage while creating system vms.****
>
> ****
>
> Thanks,****
>
> -Nitin****
>
> ****
>
> *From:* Lu Heng [mailto:h.lu@anytimechinese.com]
> *Sent:* Tuesday, June 12, 2012 3:27 PM
> *To:* Nitin Mehta
> *Cc:* cloudstack-dev@incubator.apache.org****
>
>
> *Subject:* Re: Secondary storage is not working****
>
> ****
>
> Hi****
>
> On Tue, Jun 12, 2012 at 6:02 AM, Nitin Mehta <Ni...@citrix.com>
> wrote:****
>
> Lu - With the information provided it seems that the system vm template
> and secondary storage have been configured correctly.
> Let's circle back to the error in the logs then. Can you try and manually
> mount the entire path (1.1.128.2:/data/template/tmpl/1/1) on the
> xenserver ?****
>
> ****
>
> I tried, it works. ****
>
> com.cloud.utils.exception.CloudRuntimeException: can't mount
> 1.1.128.2:/data/template/tmpl/1/1****
>
> to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde ?
>
> If this works then let's look into the logs of this failure which should
> be available on SMlog on the hypervisor. Search for the mounting error and
> see if you can find something there.****
>
> SMlog, how can I find it? ****
>
>
> Thanks,
> -Nitin
>
> -----Original Message-----
> From: Lu Heng [mailto:h.lu@anytimechinese.com]****
>
> Sent: Tuesday, June 12, 2012 9:19 AM
> To: cloudstack-dev@incubator.apache.org
> Cc: Nitin Mehta
> Subject: Re: Secondary storage is not working
>
> Hi****
>
> some more information:
>
> Server IP:1.1.1.1(replaced with example ip, but the path is the same)
> path:/data
>
> preparation of the system template:
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /data -u
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> -h<http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2%0d%0a-h>xenserver -F
> --2012-06-11 23:44:07--
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> Resolving download.cloud.com... 207.171.185.201 Connecting to
> download.cloud.com|207.171.185.201|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 140616708 (134M) [binary/octet-stream] Saving to:
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
>
>
> 100%[==================================================================================================================================>]
> 140,616,708 42.4M/s in 4.7s
>
> 2012-06-11 23:44:12 (28.3 MB/s) -
>
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
> saved [140616708/140616708]
>
> Uncompressing to
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
> (type bz2)...could take a long time
> Moving to
> /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
> take a while
> Successfully installed system VM template to /data/template/tmpl/1/1/
>
> /etc/exports:
> /data *(rw,async,no_root_squash)
>
> Please let me know if you need more information.
> On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h....@anytimechinese.com> wrote:
> > Hi Nitin:
> >
> > Thanks for replying.
> >
> > See my reply.
> >
> > On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com>
> wrote:
> >>
> >> Lu,
> >>
> >> Can you paste the snippet of logs where you see the error? Can you
> please elaborate on what you mean by secondary storage setup is not working.
> >> In my knowledge, MS doesn't validate the secondary storage path and
> does very basic tests while adding it. Typically I have seen the following
> issues users commit while adding secondary storage.
> >> Please check if your issue falls under one of them.
> >>
> >> 1. Users skip the preparation of the storage (e.g. not seeding
> >> the System VM to the NFS export or seeding the wrong version for the
> >> hypervisor)
> >
> > /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmp
> > lt -m /mnt/secondary -u
> > http://download.cloud.com/templates/acton/acton-systemvm-
> > 02062012.vhd.bz2 -h xenserver -F
> > is proformed in the management server, in which
> > /data/template/tmpl/1/1 was created, and a vhd file together with a
> > template.properties was under that folder.
> > three things I saw:
> > 1. under template section of the cloudstack, I saw SystemVM Template
> > (XenServer) usdatacenter1 XenServer, and if I open it, I saw
> > "download complete"
> > However, seems it have nothing to do with the real storage, as if I
> > delete the secondary storage, it is still there.
> > 2. in the installation guide it says it tooks up to 30 mins to
> > download that template, however, I saw "successful installation" after
> > about 30 seconds.
> > 3. seems cloud stack does not check the secondary storage, i tried to
> > add random IP address with random path, it still shows "enabled".
> >
> >>
> >> 2. NFS v4 is used which causes errors during Secondary Storage
> >> preparation (Solution: mount the storage and force NFS v3 before
> >> preparing the Storage: mount -v -t nfs -o vers=3
> >> servername:/nfs/share /mnt/secondary)
> > output from the management server:
> > Server rpc stats:
> > calls badcalls badauth badclnt xdrcall
> > 3803 0 0 0 0
> >
> > Server nfs v3:
> > null getattr setattr lookup access
> > readlink
> > 3232 85% 12 0% 0 0% 0 0% 5 0% 0
>
> > 0% read write create mkdir symlink
> > mknod
> > 545 14% 0 0% 0 0% 0 0% 0 0% 0
>
> > 0% remove rmdir rename link readdir
> > readdirplus
> > 0 0% 0 0% 0 0% 0 0% 0 0% 3
>
> > 0% fsstat fsinfo pathconf commit
> > 0 0% 4 0% 0 0% 0 0%
> >
> >> 3. The /etc/export file on the NFS does not contain
> 'no_root_squash'
> > It does.
> >> 4. The /etc/export file on the NFS blocks the host/Management
> >> Network CIDR
> > the NFS is on the management server.
> >> 5. Users enter wrong NFS share path (no validation by
> >> CloudStack)
> > it is correct as I have successfully mounted it from Host sever.
> >>
> >> Thanks,
> >> -Nitin
> >>
> >> -----Original Message-----
> >> From: Lu Heng [mailto:h.lu@anytimechinese.com]
> >> Sent: Tuesday, June 12, 2012 7:59 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Secondary storage is not working
> >>
> >> Hi
> >>
> >> I have secondary storage setup at management server's /data path, and
> while I add it to the zone, it is not working(with information
> server:<management-server-IP> path:/data), however,I can successfully mount
> secondary storage in the HV server, any one have idea what is wrong?
> >>
> >>
> >> --
> >> --
> >> Kind regards.
> >> Lu
> >>
> >> This transmission is intended solely for the addressee(s) shown above.
> >> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use of
> this transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.
> >
> >
> >
> >
> > --
> > --
> > Kind regards.
> > Lu
> >
> > This transmission is intended solely for the addressee(s) shown above.
> > It may contain information that is privileged, confidential or
> > otherwise protected from disclosure. Any review, dissemination or use
> > of this transmission or its contents by persons other than the
> > intended addressee(s) is strictly prohibited. If you have received
> > this transmission in error, please notify this office immediately and
> > e-mail the original at the sender's address above by replying to this
> > message and including the text of the transmission received.
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or otherwise
> protected from disclosure. Any review, dissemination or use of this
> transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.****
>
>
>
> ****
>
> ****
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.****
>
>
>
> ****
>
> ** **
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.****
>
>
>
> ****
>
> ** **
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.****
>
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
Re: Secondary storage is not working
Posted by Lu Heng <h....@anytimechinese.com>.
Hi
I didn't see anything interesting here. Here is the most recent version(but
I have looked at the rotated vew ones, looks the same)
[13125] 2012-06-12 05:01:41.648419 #### VMOPS enter get_rule_logs_for_vms
####
[13125] 2012-06-12 05:01:41.683571 #### VMOPS enter
network_rules_for_rebooted_vm ####
[13125] 2012-06-12 05:01:41.683614 #### VMOPS enter check_domid_changed
####
[13125] 2012-06-12 05:01:41.693652 #### VMOPS exit check_domid_changed ####
[13125] 2012-06-12 05:01:41.693690 #### VMOPS exit
network_rules_for_rebooted_vm ####
[13125] 2012-06-12 05:01:41.693732 #### VMOPS exit get_rule_logs_for_vms
####
[13181] 2012-06-12 05:01:49.672472 #### VMOPS enter gethostvmstats ####
[13181] 2012-06-12 05:01:49.680152 #### VMOPS exit gethostvmstats ####
[13189] 2012-06-12 05:01:50.688317 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13189] 2012-06-12 05:01:50.696646 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13189] 2012-06-12 05:01:50.734480 lock: acquired
/var/lock/sm/iscsiadm/running
[13189] 2012-06-12 05:01:50.746575 lock: released
/var/lock/sm/iscsiadm/running
[13189] 2012-06-12 05:01:50.746687 lock: closed
/var/lock/sm/iscsiadm/running
[13189] 2012-06-12 05:01:50.747452 ['ls', '/sys/class/scsi_host', '-1',
'--color=never']
[13189] 2012-06-12 05:01:50.759330 SUCCESS
[13189] 2012-06-12 05:01:50.759950 ['11']
[13189] 2012-06-12 05:01:50.763759 PATHDICT: key 10.2.0.5:3260: {'path':
'/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
'ipaddr': '10.2.0.5', 'port': 3260}
[13189] 2012-06-12 05:01:50.763857 Path found:
/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260
[13189] 2012-06-12 05:01:50.777597 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13189] 2012-06-12 05:01:50.792979 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.815790 SUCCESS
[13189] 2012-06-12 05:01:50.816004 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.816106 LVMCache: will initialize now
[13189] 2012-06-12 05:01:50.816175 LVMCache: refreshing
[13189] 2012-06-12 05:01:50.816269 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.839279 SUCCESS
[13189] 2012-06-12 05:01:50.839672 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.839784 Entering _checkMetadataVolume
[13189] 2012-06-12 05:01:50.841193 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.841309 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13189] 2012-06-12 05:01:50.856727 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.878072 SUCCESS
[13189] 2012-06-12 05:01:50.878301 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.878406 LVMCache: will initialize now
[13189] 2012-06-12 05:01:50.878475 LVMCache: refreshing
[13189] 2012-06-12 05:01:50.878568 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.900543 SUCCESS
[13189] 2012-06-12 05:01:50.900936 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.901037 Entering _checkMetadataVolume
[13189] 2012-06-12 05:01:50.901347 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.901531 sr_scan {'sr_uuid':
'900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
'OpaqueRef:ebe4d090-bf89-d8d4-c200-591d95395fa5', 'args': [], 'host_ref':
'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
'OpaqueRef:d89486fe-07fc-182b-0ba9-f770211adfdc', 'device_config':
{'device':
'/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
'10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}
[13189] 2012-06-12 05:01:50.901698 LVHDSR.scan for
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13189] 2012-06-12 05:01:50.901922 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.943280 SUCCESS
[13189] 2012-06-12 05:01:50.944136 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.965564 SUCCESS
[13189] 2012-06-12 05:01:51.038455 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
(exists: True)
[13189] 2012-06-12 05:01:51.038507 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13189] 2012-06-12 05:01:51.038537 Kicking GC
<13189> 2012-06-12 05:01:51.038575 === SR
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===
<13223> 2012-06-12 05:01:51.040066 Will finish as PID [13224]
<13189> 2012-06-12 05:01:51.040723 New PID [13223]
[13189] 2012-06-12 05:01:51.040941 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13189] 2012-06-12 05:01:51.041280 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:51.047912 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13224] 2012-06-12 05:01:51.094792 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13224] 2012-06-12 05:01:51.112970 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
(exists: True)
[13224] 2012-06-12 05:01:51.113013 LVMCache: refreshing
[13224] 2012-06-12 05:01:51.113050 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.136424 SUCCESS
[13224] 2012-06-12 05:01:51.136869 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.179332 SUCCESS
<13224> 2012-06-12 05:01:51.180168 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees):
*0de05e6d[VHD](2.000G//540.000M|ao)
fedba9ba[VHD](2.000G//2.012G|ao)
5bca5a2b[VHD](2.000G//2.012G|n)
bf63b940[VHD](2.000G//8.000M|n)
[13224] 2012-06-12 05:01:51.180354 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13224] 2012-06-12 05:01:51.180523 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.201612 SUCCESS
[13224] 2012-06-12 05:01:51.201856 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.223280 SUCCESS
<13224> 2012-06-12 05:01:51.238128 Got sm-config for
*0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
<13224> 2012-06-12 05:01:51.238235 No work, exiting
<13224> 2012-06-12 05:01:51.238473 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
[13367] 2012-06-12 05:02:41.632069 #### VMOPS enter get_rule_logs_for_vms
####
[13367] 2012-06-12 05:02:41.657769 #### VMOPS enter
network_rules_for_rebooted_vm ####
[13367] 2012-06-12 05:02:41.657863 #### VMOPS enter check_domid_changed
####
[13367] 2012-06-12 05:02:41.669337 #### VMOPS exit check_domid_changed ####
[13367] 2012-06-12 05:02:41.669374 #### VMOPS exit
network_rules_for_rebooted_vm ####
[13367] 2012-06-12 05:02:41.669416 #### VMOPS exit get_rule_logs_for_vms
####
[13412] 2012-06-12 05:02:50.192681 #### VMOPS enter gethostvmstats ####
[13412] 2012-06-12 05:02:50.200365 #### VMOPS exit gethostvmstats ####
[13420] 2012-06-12 05:02:51.214194 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13420] 2012-06-12 05:02:51.222629 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13420] 2012-06-12 05:02:51.260176 lock: acquired
/var/lock/sm/iscsiadm/running
[13420] 2012-06-12 05:02:51.271762 lock: released
/var/lock/sm/iscsiadm/running
[13420] 2012-06-12 05:02:51.271872 lock: closed
/var/lock/sm/iscsiadm/running
[13420] 2012-06-12 05:02:51.272634 ['ls', '/sys/class/scsi_host', '-1',
'--color=never']
[13420] 2012-06-12 05:02:51.284386 SUCCESS
[13420] 2012-06-12 05:02:51.285007 ['11']
[13420] 2012-06-12 05:02:51.288837 PATHDICT: key 10.2.0.5:3260: {'path':
'/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
'ipaddr': '10.2.0.5', 'port': 3260}
[13420] 2012-06-12 05:02:51.288936 Path found:
/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260
[13420] 2012-06-12 05:02:51.302907 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13420] 2012-06-12 05:02:51.318194 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.339945 SUCCESS
[13420] 2012-06-12 05:02:51.340163 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.340265 LVMCache: will initialize now
[13420] 2012-06-12 05:02:51.340334 LVMCache: refreshing
[13420] 2012-06-12 05:02:51.340431 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.363255 SUCCESS
[13420] 2012-06-12 05:02:51.363653 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.363758 Entering _checkMetadataVolume
[13420] 2012-06-12 05:02:51.365197 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.365314 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13420] 2012-06-12 05:02:51.380298 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.401257 SUCCESS
[13420] 2012-06-12 05:02:51.401471 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.401572 LVMCache: will initialize now
[13420] 2012-06-12 05:02:51.401641 LVMCache: refreshing
[13420] 2012-06-12 05:02:51.401734 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.424356 SUCCESS
[13420] 2012-06-12 05:02:51.424750 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.424856 Entering _checkMetadataVolume
[13420] 2012-06-12 05:02:51.425167 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.425353 sr_scan {'sr_uuid':
'900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
'OpaqueRef:672a5cf7-d938-c6bd-001f-7b59d4a69204', 'args': [], 'host_ref':
'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
'OpaqueRef:af2690fc-6539-daf2-ff58-30fe2f9a0371', 'device_config':
{'device':
'/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
'10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}
[13420] 2012-06-12 05:02:51.425503 LVHDSR.scan for
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13420] 2012-06-12 05:02:51.425726 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.468757 SUCCESS
[13420] 2012-06-12 05:02:51.469610 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.491025 SUCCESS
[13420] 2012-06-12 05:02:51.564940 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
(exists: True)
[13420] 2012-06-12 05:02:51.564993 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13420] 2012-06-12 05:02:51.565033 Kicking GC
<13420> 2012-06-12 05:02:51.565073 === SR
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===
<13454> 2012-06-12 05:02:51.566603 Will finish as PID [13455]
<13420> 2012-06-12 05:02:51.567259 New PID [13454]
[13420] 2012-06-12 05:02:51.567476 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13420] 2012-06-12 05:02:51.567812 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.574404 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13455] 2012-06-12 05:02:51.621214 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13455] 2012-06-12 05:02:51.639586 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
(exists: True)
[13455] 2012-06-12 05:02:51.639629 LVMCache: refreshing
[13455] 2012-06-12 05:02:51.639667 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.662316 SUCCESS
[13455] 2012-06-12 05:02:51.662762 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.704936 SUCCESS
<13455> 2012-06-12 05:02:51.705771 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees):
*0de05e6d[VHD](2.000G//540.000M|ao)
fedba9ba[VHD](2.000G//2.012G|ao)
5bca5a2b[VHD](2.000G//2.012G|n)
bf63b940[VHD](2.000G//8.000M|n)
[13455] 2012-06-12 05:02:51.705959 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13455] 2012-06-12 05:02:51.706127 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.727751 SUCCESS
[13455] 2012-06-12 05:02:51.727992 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.748946 SUCCESS
<13455> 2012-06-12 05:02:51.761594 Got sm-config for
*0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
<13455> 2012-06-12 05:02:51.761638 No work, exiting
<13455> 2012-06-12 05:02:51.761737 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
[13614] 2012-06-12 05:03:41.631406 #### VMOPS enter get_rule_logs_for_vms
####
[13614] 2012-06-12 05:03:41.656623 #### VMOPS enter
network_rules_for_rebooted_vm ####
[13614] 2012-06-12 05:03:41.656718 #### VMOPS enter check_domid_changed
####
[13614] 2012-06-12 05:03:41.668513 #### VMOPS exit check_domid_changed ####
[13614] 2012-06-12 05:03:41.668551 #### VMOPS exit
network_rules_for_rebooted_vm ####
[13614] 2012-06-12 05:03:41.668593 #### VMOPS exit get_rule_logs_for_vms
####
[13659] 2012-06-12 05:03:50.674948 #### VMOPS enter gethostvmstats ####
[13659] 2012-06-12 05:03:50.682474 #### VMOPS exit gethostvmstats ####
[13669] 2012-06-12 05:03:51.816879 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13669] 2012-06-12 05:03:51.825230 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13669] 2012-06-12 05:03:51.861581 lock: acquired
/var/lock/sm/iscsiadm/running
[13669] 2012-06-12 05:03:51.873663 lock: released
/var/lock/sm/iscsiadm/running
[13669] 2012-06-12 05:03:51.873776 lock: closed
/var/lock/sm/iscsiadm/running
[13669] 2012-06-12 05:03:51.874533 ['ls', '/sys/class/scsi_host', '-1',
'--color=never']
[13669] 2012-06-12 05:03:51.884299 SUCCESS
[13669] 2012-06-12 05:03:51.884918 ['11']
[13669] 2012-06-12 05:03:51.888744 PATHDICT: key 10.2.0.5:3260: {'path':
'/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
'ipaddr': '10.2.0.5', 'port': 3260}
[13669] 2012-06-12 05:03:51.888843 Path found:
/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260
[13669] 2012-06-12 05:03:51.902943 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13669] 2012-06-12 05:03:51.918180 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:51.940914 SUCCESS
[13669] 2012-06-12 05:03:51.941132 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:51.941234 LVMCache: will initialize now
[13669] 2012-06-12 05:03:51.941304 LVMCache: refreshing
[13669] 2012-06-12 05:03:51.941401 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:51.964523 SUCCESS
[13669] 2012-06-12 05:03:51.964912 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:51.965019 Entering _checkMetadataVolume
[13669] 2012-06-12 05:03:51.966443 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:51.966560 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13669] 2012-06-12 05:03:51.982203 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.003569 SUCCESS
[13669] 2012-06-12 05:03:52.003665 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.003711 LVMCache: will initialize now
[13669] 2012-06-12 05:03:52.003741 LVMCache: refreshing
[13669] 2012-06-12 05:03:52.003784 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.025826 SUCCESS
[13669] 2012-06-12 05:03:52.026232 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.026336 Entering _checkMetadataVolume
[13669] 2012-06-12 05:03:52.026648 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.026834 sr_scan {'sr_uuid':
'900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
'OpaqueRef:70e76ca3-6e80-c865-98bb-9dbfa362379a', 'args': [], 'host_ref':
'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
'OpaqueRef:2aa1c42e-3b05-c271-bf60-57428a855e46', 'device_config':
{'device':
'/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
'10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}
[13669] 2012-06-12 05:03:52.026984 LVHDSR.scan for
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13669] 2012-06-12 05:03:52.027207 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.070068 SUCCESS
[13669] 2012-06-12 05:03:52.070933 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.092699 SUCCESS
[13669] 2012-06-12 05:03:52.160000 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
(exists: True)
[13669] 2012-06-12 05:03:52.160052 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13669] 2012-06-12 05:03:52.160082 Kicking GC
<13669> 2012-06-12 05:03:52.160121 === SR
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===
<13703> 2012-06-12 05:03:52.161610 Will finish as PID [13704]
<13669> 2012-06-12 05:03:52.162260 New PID [13703]
[13669] 2012-06-12 05:03:52.162484 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13669] 2012-06-12 05:03:52.162823 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.169460 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13704] 2012-06-12 05:03:52.221888 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13704] 2012-06-12 05:03:52.256927 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
(exists: True)
[13704] 2012-06-12 05:03:52.257025 LVMCache: refreshing
[13704] 2012-06-12 05:03:52.257133 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.282878 SUCCESS
[13704] 2012-06-12 05:03:52.283329 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.332991 SUCCESS
<13704> 2012-06-12 05:03:52.333830 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees):
*0de05e6d[VHD](2.000G//540.000M|ao)
fedba9ba[VHD](2.000G//2.012G|ao)
5bca5a2b[VHD](2.000G//2.012G|n)
bf63b940[VHD](2.000G//8.000M|n)
[13704] 2012-06-12 05:03:52.334019 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13704] 2012-06-12 05:03:52.334186 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.354072 SUCCESS
[13704] 2012-06-12 05:03:52.354315 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.376106 SUCCESS
<13704> 2012-06-12 05:03:52.393268 Got sm-config for
*0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
<13704> 2012-06-12 05:03:52.393377 No work, exiting
<13704> 2012-06-12 05:03:52.393603 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
On Tue, Jun 12, 2012 at 1:21 PM, Nitin Mehta <Ni...@citrix.com> wrote:
> I am assuming you are using Xenserver. Log in to the hypervisor. “cd
> /var/log/”. Here you can find the logs. These logs get rotated so be sure
> to search in all of them.****
>
> I am asking to you to look here because the operation
> PrimaryStorageDownloadCommand is failing which downloads the template from
> secondary to primary storage while creating system vms.****
>
> ** **
>
> Thanks,****
>
> -Nitin****
>
> ** **
>
> *From:* Lu Heng [mailto:h.lu@anytimechinese.com]
> *Sent:* Tuesday, June 12, 2012 3:27 PM
> *To:* Nitin Mehta
> *Cc:* cloudstack-dev@incubator.apache.org
>
> *Subject:* Re: Secondary storage is not working****
>
> ** **
>
> Hi****
>
> On Tue, Jun 12, 2012 at 6:02 AM, Nitin Mehta <Ni...@citrix.com>
> wrote:****
>
> Lu - With the information provided it seems that the system vm template
> and secondary storage have been configured correctly.
> Let's circle back to the error in the logs then. Can you try and manually
> mount the entire path (1.1.128.2:/data/template/tmpl/1/1) on the
> xenserver ?****
>
> ** **
>
> I tried, it works. ****
>
> com.cloud.utils.exception.CloudRuntimeException: can't mount
> 1.1.128.2:/data/template/tmpl/1/1****
>
> to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde ?
>
> If this works then let's look into the logs of this failure which should
> be available on SMlog on the hypervisor. Search for the mounting error and
> see if you can find something there.****
>
> SMlog, how can I find it? ****
>
>
> Thanks,
> -Nitin
>
> -----Original Message-----
> From: Lu Heng [mailto:h.lu@anytimechinese.com]****
>
> Sent: Tuesday, June 12, 2012 9:19 AM
> To: cloudstack-dev@incubator.apache.org
> Cc: Nitin Mehta
> Subject: Re: Secondary storage is not working
>
> Hi****
>
> some more information:
>
> Server IP:1.1.1.1(replaced with example ip, but the path is the same)
> path:/data
>
> preparation of the system template:
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /data -u
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> -h<http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2%0d%0a-h>xenserver -F
> --2012-06-11 23:44:07--
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> Resolving download.cloud.com... 207.171.185.201 Connecting to
> download.cloud.com|207.171.185.201|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 140616708 (134M) [binary/octet-stream] Saving to:
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
>
>
> 100%[==================================================================================================================================>]
> 140,616,708 42.4M/s in 4.7s
>
> 2012-06-11 23:44:12 (28.3 MB/s) -
>
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
> saved [140616708/140616708]
>
> Uncompressing to
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
> (type bz2)...could take a long time
> Moving to
> /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
> take a while
> Successfully installed system VM template to /data/template/tmpl/1/1/
>
> /etc/exports:
> /data *(rw,async,no_root_squash)
>
> Please let me know if you need more information.
> On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h....@anytimechinese.com> wrote:
> > Hi Nitin:
> >
> > Thanks for replying.
> >
> > See my reply.
> >
> > On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com>
> wrote:
> >>
> >> Lu,
> >>
> >> Can you paste the snippet of logs where you see the error? Can you
> please elaborate on what you mean by secondary storage setup is not working.
> >> In my knowledge, MS doesn't validate the secondary storage path and
> does very basic tests while adding it. Typically I have seen the following
> issues users commit while adding secondary storage.
> >> Please check if your issue falls under one of them.
> >>
> >> 1. Users skip the preparation of the storage (e.g. not seeding
> >> the System VM to the NFS export or seeding the wrong version for the
> >> hypervisor)
> >
> > /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmp
> > lt -m /mnt/secondary -u
> > http://download.cloud.com/templates/acton/acton-systemvm-
> > 02062012.vhd.bz2 -h xenserver -F
> > is proformed in the management server, in which
> > /data/template/tmpl/1/1 was created, and a vhd file together with a
> > template.properties was under that folder.
> > three things I saw:
> > 1. under template section of the cloudstack, I saw SystemVM Template
> > (XenServer) usdatacenter1 XenServer, and if I open it, I saw
> > "download complete"
> > However, seems it have nothing to do with the real storage, as if I
> > delete the secondary storage, it is still there.
> > 2. in the installation guide it says it tooks up to 30 mins to
> > download that template, however, I saw "successful installation" after
> > about 30 seconds.
> > 3. seems cloud stack does not check the secondary storage, i tried to
> > add random IP address with random path, it still shows "enabled".
> >
> >>
> >> 2. NFS v4 is used which causes errors during Secondary Storage
> >> preparation (Solution: mount the storage and force NFS v3 before
> >> preparing the Storage: mount -v -t nfs -o vers=3
> >> servername:/nfs/share /mnt/secondary)
> > output from the management server:
> > Server rpc stats:
> > calls badcalls badauth badclnt xdrcall
> > 3803 0 0 0 0
> >
> > Server nfs v3:
> > null getattr setattr lookup access
> > readlink
> > 3232 85% 12 0% 0 0% 0 0% 5 0% 0
>
> > 0% read write create mkdir symlink
> > mknod
> > 545 14% 0 0% 0 0% 0 0% 0 0% 0
>
> > 0% remove rmdir rename link readdir
> > readdirplus
> > 0 0% 0 0% 0 0% 0 0% 0 0% 3
>
> > 0% fsstat fsinfo pathconf commit
> > 0 0% 4 0% 0 0% 0 0%
> >
> >> 3. The /etc/export file on the NFS does not contain
> 'no_root_squash'
> > It does.
> >> 4. The /etc/export file on the NFS blocks the host/Management
> >> Network CIDR
> > the NFS is on the management server.
> >> 5. Users enter wrong NFS share path (no validation by
> >> CloudStack)
> > it is correct as I have successfully mounted it from Host sever.
> >>
> >> Thanks,
> >> -Nitin
> >>
> >> -----Original Message-----
> >> From: Lu Heng [mailto:h.lu@anytimechinese.com]
> >> Sent: Tuesday, June 12, 2012 7:59 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Secondary storage is not working
> >>
> >> Hi
> >>
> >> I have secondary storage setup at management server's /data path, and
> while I add it to the zone, it is not working(with information
> server:<management-server-IP> path:/data), however,I can successfully mount
> secondary storage in the HV server, any one have idea what is wrong?
> >>
> >>
> >> --
> >> --
> >> Kind regards.
> >> Lu
> >>
> >> This transmission is intended solely for the addressee(s) shown above.
> >> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use of
> this transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.
> >
> >
> >
> >
> > --
> > --
> > Kind regards.
> > Lu
> >
> > This transmission is intended solely for the addressee(s) shown above.
> > It may contain information that is privileged, confidential or
> > otherwise protected from disclosure. Any review, dissemination or use
> > of this transmission or its contents by persons other than the
> > intended addressee(s) is strictly prohibited. If you have received
> > this transmission in error, please notify this office immediately and
> > e-mail the original at the sender's address above by replying to this
> > message and including the text of the transmission received.
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or otherwise
> protected from disclosure. Any review, dissemination or use of this
> transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.****
>
>
>
> ****
>
> ** **
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.****
>
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
RE: Secondary storage is not working
Posted by Nitin Mehta <Ni...@citrix.com>.
I am assuming you are using Xenserver. Log in to the hypervisor. "cd /var/log/". Here you can find the logs. These logs get rotated so be sure to search in all of them.
I am asking to you to look here because the operation PrimaryStorageDownloadCommand is failing which downloads the template from secondary to primary storage while creating system vms.
Thanks,
-Nitin
From: Lu Heng [mailto:h.lu@anytimechinese.com]
Sent: Tuesday, June 12, 2012 3:27 PM
To: Nitin Mehta
Cc: cloudstack-dev@incubator.apache.org
Subject: Re: Secondary storage is not working
Hi
On Tue, Jun 12, 2012 at 6:02 AM, Nitin Mehta <Ni...@citrix.com>> wrote:
Lu - With the information provided it seems that the system vm template and secondary storage have been configured correctly.
Let's circle back to the error in the logs then. Can you try and manually mount the entire path (1.1.128.2:/data/template/tmpl/1/1) on the xenserver ?
I tried, it works.
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde ?
If this works then let's look into the logs of this failure which should be available on SMlog on the hypervisor. Search for the mounting error and see if you can find something there.
SMlog, how can I find it?
Thanks,
-Nitin
-----Original Message-----
From: Lu Heng [mailto:h.lu@anytimechinese.com<ma...@anytimechinese.com>]
Sent: Tuesday, June 12, 2012 9:19 AM
To: cloudstack-dev@incubator.apache.org<ma...@incubator.apache.org>
Cc: Nitin Mehta
Subject: Re: Secondary storage is not working
Hi
some more information:
Server IP:1.1.1.1(replaced with example ip, but the path is the same) path:/data
preparation of the system template:
/usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
-m /data -u http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
-h<http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2%0d%0a-h> xenserver -F
--2012-06-11 23:44:07--
http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
Resolving download.cloud.com... 207.171.185.201 Connecting to download.cloud.com<http://download.cloud.com>|207.171.185.201|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 140616708 (134M) [binary/octet-stream] Saving to: `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
100%[==================================================================================================================================>]
140,616,708 42.4M/s in 4.7s
2012-06-11 23:44:12 (28.3 MB/s) -
`/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
saved [140616708/140616708]
Uncompressing to
/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
(type bz2)...could take a long time
Moving to /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
take a while
Successfully installed system VM template to /data/template/tmpl/1/1/
/etc/exports:
/data *(rw,async,no_root_squash)
Please let me know if you need more information.
On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h....@anytimechinese.com>> wrote:
> Hi Nitin:
>
> Thanks for replying.
>
> See my reply.
>
> On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com>> wrote:
>>
>> Lu,
>>
>> Can you paste the snippet of logs where you see the error? Can you please elaborate on what you mean by secondary storage setup is not working.
>> In my knowledge, MS doesn't validate the secondary storage path and does very basic tests while adding it. Typically I have seen the following issues users commit while adding secondary storage.
>> Please check if your issue falls under one of them.
>>
>> 1. Users skip the preparation of the storage (e.g. not seeding
>> the System VM to the NFS export or seeding the wrong version for the
>> hypervisor)
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmp
> lt -m /mnt/secondary -u
> http://download.cloud.com/templates/acton/acton-systemvm-
> 02062012.vhd.bz2 -h xenserver -F
> is proformed in the management server, in which
> /data/template/tmpl/1/1 was created, and a vhd file together with a
> template.properties was under that folder.
> three things I saw:
> 1. under template section of the cloudstack, I saw SystemVM Template
> (XenServer) usdatacenter1 XenServer, and if I open it, I saw
> "download complete"
> However, seems it have nothing to do with the real storage, as if I
> delete the secondary storage, it is still there.
> 2. in the installation guide it says it tooks up to 30 mins to
> download that template, however, I saw "successful installation" after
> about 30 seconds.
> 3. seems cloud stack does not check the secondary storage, i tried to
> add random IP address with random path, it still shows "enabled".
>
>>
>> 2. NFS v4 is used which causes errors during Secondary Storage
>> preparation (Solution: mount the storage and force NFS v3 before
>> preparing the Storage: mount -v -t nfs -o vers=3
>> servername:/nfs/share /mnt/secondary)
> output from the management server:
> Server rpc stats:
> calls badcalls badauth badclnt xdrcall
> 3803 0 0 0 0
>
> Server nfs v3:
> null getattr setattr lookup access
> readlink
> 3232 85% 12 0% 0 0% 0 0% 5 0% 0
> 0% read write create mkdir symlink
> mknod
> 545 14% 0 0% 0 0% 0 0% 0 0% 0
> 0% remove rmdir rename link readdir
> readdirplus
> 0 0% 0 0% 0 0% 0 0% 0 0% 3
> 0% fsstat fsinfo pathconf commit
> 0 0% 4 0% 0 0% 0 0%
>
>> 3. The /etc/export file on the NFS does not contain 'no_root_squash'
> It does.
>> 4. The /etc/export file on the NFS blocks the host/Management
>> Network CIDR
> the NFS is on the management server.
>> 5. Users enter wrong NFS share path (no validation by
>> CloudStack)
> it is correct as I have successfully mounted it from Host sever.
>>
>> Thanks,
>> -Nitin
>>
>> -----Original Message-----
>> From: Lu Heng [mailto:h.lu@anytimechinese.com<ma...@anytimechinese.com>]
>> Sent: Tuesday, June 12, 2012 7:59 AM
>> To: cloudstack-dev@incubator.apache.org<ma...@incubator.apache.org>
>> Subject: Secondary storage is not working
>>
>> Hi
>>
>> I have secondary storage setup at management server's /data path, and while I add it to the zone, it is not working(with information server:<management-server-IP> path:/data), however,I can successfully mount secondary storage in the HV server, any one have idea what is wrong?
>>
>>
>> --
>> --
>> Kind regards.
>> Lu
>>
>> This transmission is intended solely for the addressee(s) shown above.
>> It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
>
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
Re: Secondary storage is not working
Posted by Lu Heng <h....@anytimechinese.com>.
Hi
On Tue, Jun 12, 2012 at 6:02 AM, Nitin Mehta <Ni...@citrix.com> wrote:
> Lu - With the information provided it seems that the system vm template
> and secondary storage have been configured correctly.
> Let's circle back to the error in the logs then. Can you try and manually
> mount the entire path (1.1.128.2:/data/template/tmpl/1/1) on the
> xenserver ?
>
> I tried, it works.
> com.cloud.utils.exception.CloudRuntimeException: can't mount
> 1.1.128.2:/data/template/tmpl/1/1
> to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde ?
>
> If this works then let's look into the logs of this failure which should
> be available on SMlog on the hypervisor. Search for the mounting error and
> see if you can find something there.
>
SMlog, how can I find it?
>
> Thanks,
> -Nitin
>
> -----Original Message-----
> From: Lu Heng [mailto:h.lu@anytimechinese.com]
> Sent: Tuesday, June 12, 2012 9:19 AM
> To: cloudstack-dev@incubator.apache.org
> Cc: Nitin Mehta
> Subject: Re: Secondary storage is not working
>
> Hi
>
> some more information:
>
> Server IP:1.1.1.1(replaced with example ip, but the path is the same)
> path:/data
>
> preparation of the system template:
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /data -u
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> -h xenserver -F
> --2012-06-11 23:44:07--
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> Resolving download.cloud.com... 207.171.185.201 Connecting to
> download.cloud.com|207.171.185.201|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 140616708 (134M) [binary/octet-stream] Saving to:
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
>
>
> 100%[==================================================================================================================================>]
> 140,616,708 42.4M/s in 4.7s
>
> 2012-06-11 23:44:12 (28.3 MB/s) -
>
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
> saved [140616708/140616708]
>
> Uncompressing to
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
> (type bz2)...could take a long time
> Moving to
> /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
> take a while
> Successfully installed system VM template to /data/template/tmpl/1/1/
>
> /etc/exports:
> /data *(rw,async,no_root_squash)
>
> Please let me know if you need more information.
> On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h....@anytimechinese.com> wrote:
> > Hi Nitin:
> >
> > Thanks for replying.
> >
> > See my reply.
> >
> > On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com>
> wrote:
> >>
> >> Lu,
> >>
> >> Can you paste the snippet of logs where you see the error? Can you
> please elaborate on what you mean by secondary storage setup is not working.
> >> In my knowledge, MS doesn't validate the secondary storage path and
> does very basic tests while adding it. Typically I have seen the following
> issues users commit while adding secondary storage.
> >> Please check if your issue falls under one of them.
> >>
> >> 1. Users skip the preparation of the storage (e.g. not seeding
> >> the System VM to the NFS export or seeding the wrong version for the
> >> hypervisor)
> >
> > /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmp
> > lt -m /mnt/secondary -u
> > http://download.cloud.com/templates/acton/acton-systemvm-
> > 02062012.vhd.bz2 -h xenserver -F
> > is proformed in the management server, in which
> > /data/template/tmpl/1/1 was created, and a vhd file together with a
> > template.properties was under that folder.
> > three things I saw:
> > 1. under template section of the cloudstack, I saw SystemVM Template
> > (XenServer) usdatacenter1 XenServer, and if I open it, I saw
> > "download complete"
> > However, seems it have nothing to do with the real storage, as if I
> > delete the secondary storage, it is still there.
> > 2. in the installation guide it says it tooks up to 30 mins to
> > download that template, however, I saw "successful installation" after
> > about 30 seconds.
> > 3. seems cloud stack does not check the secondary storage, i tried to
> > add random IP address with random path, it still shows "enabled".
> >
> >>
> >> 2. NFS v4 is used which causes errors during Secondary Storage
> >> preparation (Solution: mount the storage and force NFS v3 before
> >> preparing the Storage: mount -v -t nfs -o vers=3
> >> servername:/nfs/share /mnt/secondary)
> > output from the management server:
> > Server rpc stats:
> > calls badcalls badauth badclnt xdrcall
> > 3803 0 0 0 0
> >
> > Server nfs v3:
> > null getattr setattr lookup access
> > readlink
> > 3232 85% 12 0% 0 0% 0 0% 5 0% 0
>
> > 0% read write create mkdir symlink
> > mknod
> > 545 14% 0 0% 0 0% 0 0% 0 0% 0
>
> > 0% remove rmdir rename link readdir
> > readdirplus
> > 0 0% 0 0% 0 0% 0 0% 0 0% 3
>
> > 0% fsstat fsinfo pathconf commit
> > 0 0% 4 0% 0 0% 0 0%
> >
> >> 3. The /etc/export file on the NFS does not contain
> 'no_root_squash'
> > It does.
> >> 4. The /etc/export file on the NFS blocks the host/Management
> >> Network CIDR
> > the NFS is on the management server.
> >> 5. Users enter wrong NFS share path (no validation by
> >> CloudStack)
> > it is correct as I have successfully mounted it from Host sever.
> >>
> >> Thanks,
> >> -Nitin
> >>
> >> -----Original Message-----
> >> From: Lu Heng [mailto:h.lu@anytimechinese.com]
> >> Sent: Tuesday, June 12, 2012 7:59 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Secondary storage is not working
> >>
> >> Hi
> >>
> >> I have secondary storage setup at management server's /data path, and
> while I add it to the zone, it is not working(with information
> server:<management-server-IP> path:/data), however,I can successfully mount
> secondary storage in the HV server, any one have idea what is wrong?
> >>
> >>
> >> --
> >> --
> >> Kind regards.
> >> Lu
> >>
> >> This transmission is intended solely for the addressee(s) shown above.
> >> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use of
> this transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.
> >
> >
> >
> >
> > --
> > --
> > Kind regards.
> > Lu
> >
> > This transmission is intended solely for the addressee(s) shown above.
> > It may contain information that is privileged, confidential or
> > otherwise protected from disclosure. Any review, dissemination or use
> > of this transmission or its contents by persons other than the
> > intended addressee(s) is strictly prohibited. If you have received
> > this transmission in error, please notify this office immediately and
> > e-mail the original at the sender's address above by replying to this
> > message and including the text of the transmission received.
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or otherwise
> protected from disclosure. Any review, dissemination or use of this
> transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.
>
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
RE: Secondary storage is not working
Posted by Nitin Mehta <Ni...@citrix.com>.
Lu - With the information provided it seems that the system vm template and secondary storage have been configured correctly.
Let's circle back to the error in the logs then. Can you try and manually mount the entire path (1.1.128.2:/data/template/tmpl/1/1) on the xenserver ?
com.cloud.utils.exception.CloudRuntimeException: can't mount
1.1.128.2:/data/template/tmpl/1/1
to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde ?
If this works then let's look into the logs of this failure which should be available on SMlog on the hypervisor. Search for the mounting error and see if you can find something there.
Thanks,
-Nitin
-----Original Message-----
From: Lu Heng [mailto:h.lu@anytimechinese.com]
Sent: Tuesday, June 12, 2012 9:19 AM
To: cloudstack-dev@incubator.apache.org
Cc: Nitin Mehta
Subject: Re: Secondary storage is not working
Hi
some more information:
Server IP:1.1.1.1(replaced with example ip, but the path is the same) path:/data
preparation of the system template:
/usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
-m /data -u http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
-h xenserver -F
--2012-06-11 23:44:07--
http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
Resolving download.cloud.com... 207.171.185.201 Connecting to download.cloud.com|207.171.185.201|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 140616708 (134M) [binary/octet-stream] Saving to: `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
100%[==================================================================================================================================>]
140,616,708 42.4M/s in 4.7s
2012-06-11 23:44:12 (28.3 MB/s) -
`/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
saved [140616708/140616708]
Uncompressing to
/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
(type bz2)...could take a long time
Moving to /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
take a while
Successfully installed system VM template to /data/template/tmpl/1/1/
/etc/exports:
/data *(rw,async,no_root_squash)
Please let me know if you need more information.
On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h....@anytimechinese.com> wrote:
> Hi Nitin:
>
> Thanks for replying.
>
> See my reply.
>
> On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com> wrote:
>>
>> Lu,
>>
>> Can you paste the snippet of logs where you see the error? Can you please elaborate on what you mean by secondary storage setup is not working.
>> In my knowledge, MS doesn't validate the secondary storage path and does very basic tests while adding it. Typically I have seen the following issues users commit while adding secondary storage.
>> Please check if your issue falls under one of them.
>>
>> 1. Users skip the preparation of the storage (e.g. not seeding
>> the System VM to the NFS export or seeding the wrong version for the
>> hypervisor)
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmp
> lt -m /mnt/secondary -u
> http://download.cloud.com/templates/acton/acton-systemvm-
> 02062012.vhd.bz2 -h xenserver -F
> is proformed in the management server, in which
> /data/template/tmpl/1/1 was created, and a vhd file together with a
> template.properties was under that folder.
> three things I saw:
> 1. under template section of the cloudstack, I saw SystemVM Template
> (XenServer) usdatacenter1 XenServer, and if I open it, I saw
> "download complete"
> However, seems it have nothing to do with the real storage, as if I
> delete the secondary storage, it is still there.
> 2. in the installation guide it says it tooks up to 30 mins to
> download that template, however, I saw "successful installation" after
> about 30 seconds.
> 3. seems cloud stack does not check the secondary storage, i tried to
> add random IP address with random path, it still shows "enabled".
>
>>
>> 2. NFS v4 is used which causes errors during Secondary Storage
>> preparation (Solution: mount the storage and force NFS v3 before
>> preparing the Storage: mount -v -t nfs -o vers=3
>> servername:/nfs/share /mnt/secondary)
> output from the management server:
> Server rpc stats:
> calls badcalls badauth badclnt xdrcall
> 3803 0 0 0 0
>
> Server nfs v3:
> null getattr setattr lookup access
> readlink
> 3232 85% 12 0% 0 0% 0 0% 5 0% 0
> 0% read write create mkdir symlink
> mknod
> 545 14% 0 0% 0 0% 0 0% 0 0% 0
> 0% remove rmdir rename link readdir
> readdirplus
> 0 0% 0 0% 0 0% 0 0% 0 0% 3
> 0% fsstat fsinfo pathconf commit
> 0 0% 4 0% 0 0% 0 0%
>
>> 3. The /etc/export file on the NFS does not contain 'no_root_squash'
> It does.
>> 4. The /etc/export file on the NFS blocks the host/Management
>> Network CIDR
> the NFS is on the management server.
>> 5. Users enter wrong NFS share path (no validation by
>> CloudStack)
> it is correct as I have successfully mounted it from Host sever.
>>
>> Thanks,
>> -Nitin
>>
>> -----Original Message-----
>> From: Lu Heng [mailto:h.lu@anytimechinese.com]
>> Sent: Tuesday, June 12, 2012 7:59 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Secondary storage is not working
>>
>> Hi
>>
>> I have secondary storage setup at management server's /data path, and while I add it to the zone, it is not working(with information server:<management-server-IP> path:/data), however,I can successfully mount secondary storage in the HV server, any one have idea what is wrong?
>>
>>
>> --
>> --
>> Kind regards.
>> Lu
>>
>> This transmission is intended solely for the addressee(s) shown above.
>> It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
>
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
Re: Secondary storage is not working
Posted by Lu Heng <h....@anytimechinese.com>.
Hi
some more information:
Server IP:1.1.1.1(replaced with example ip, but the path is the same)
path:/data
preparation of the system template:
/usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
-m /data -u http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
-h xenserver -F
--2012-06-11 23:44:07--
http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
Resolving download.cloud.com... 207.171.185.201
Connecting to download.cloud.com|207.171.185.201|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 140616708 (134M) [binary/octet-stream]
Saving to: `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
100%[==================================================================================================================================>]
140,616,708 42.4M/s in 4.7s
2012-06-11 23:44:12 (28.3 MB/s) -
`/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
saved [140616708/140616708]
Uncompressing to
/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
(type bz2)...could take a long time
Moving to /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
take a while
Successfully installed system VM template to /data/template/tmpl/1/1/
/etc/exports:
/data *(rw,async,no_root_squash)
Please let me know if you need more information.
On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h....@anytimechinese.com> wrote:
> Hi Nitin:
>
> Thanks for replying.
>
> See my reply.
>
> On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com> wrote:
>>
>> Lu,
>>
>> Can you paste the snippet of logs where you see the error? Can you please elaborate on what you mean by secondary storage setup is not working.
>> In my knowledge, MS doesn't validate the secondary storage path and does very basic tests while adding it. Typically I have seen the following issues users commit while adding secondary storage.
>> Please check if your issue falls under one of them.
>>
>> 1. Users skip the preparation of the storage (e.g. not seeding the System VM to the NFS export or seeding the wrong version for the hypervisor)
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt -m
> /mnt/secondary -u http://download.cloud.com/templates/acton/acton-systemvm-
> 02062012.vhd.bz2 -h xenserver -F
> is proformed in the management server, in which
> /data/template/tmpl/1/1 was created, and a vhd file together with a
> template.properties was under that folder.
> three things I saw:
> 1. under template section of the cloudstack, I saw
> SystemVM Template (XenServer) usdatacenter1 XenServer, and if I open
> it, I saw "download complete"
> However, seems it have nothing to do with the real storage, as if I
> delete the secondary storage, it is still there.
> 2. in the installation guide it says it tooks up to 30 mins to
> download that template, however, I saw "successful installation" after
> about 30 seconds.
> 3. seems cloud stack does not check the secondary storage, i tried to
> add random IP address with random path, it still shows "enabled".
>
>>
>> 2. NFS v4 is used which causes errors during Secondary Storage preparation (Solution: mount the storage and force NFS v3 before preparing the Storage: mount -v -t nfs -o vers=3 servername:/nfs/share /mnt/secondary)
> output from the management server:
> Server rpc stats:
> calls badcalls badauth badclnt xdrcall
> 3803 0 0 0 0
>
> Server nfs v3:
> null getattr setattr lookup access readlink
> 3232 85% 12 0% 0 0% 0 0% 5 0% 0 0%
> read write create mkdir symlink mknod
> 545 14% 0 0% 0 0% 0 0% 0 0% 0 0%
> remove rmdir rename link readdir readdirplus
> 0 0% 0 0% 0 0% 0 0% 0 0% 3 0%
> fsstat fsinfo pathconf commit
> 0 0% 4 0% 0 0% 0 0%
>
>> 3. The /etc/export file on the NFS does not contain 'no_root_squash'
> It does.
>> 4. The /etc/export file on the NFS blocks the host/Management Network CIDR
> the NFS is on the management server.
>> 5. Users enter wrong NFS share path (no validation by CloudStack)
> it is correct as I have successfully mounted it from Host sever.
>>
>> Thanks,
>> -Nitin
>>
>> -----Original Message-----
>> From: Lu Heng [mailto:h.lu@anytimechinese.com]
>> Sent: Tuesday, June 12, 2012 7:59 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Secondary storage is not working
>>
>> Hi
>>
>> I have secondary storage setup at management server's /data path, and while I add it to the zone, it is not working(with information server:<management-server-IP> path:/data), however,I can successfully mount secondary storage in the HV server, any one have idea what is wrong?
>>
>>
>> --
>> --
>> Kind regards.
>> Lu
>>
>> This transmission is intended solely for the addressee(s) shown above.
>> It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
>
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
Re: Secondary storage is not working
Posted by Lu Heng <h....@anytimechinese.com>.
Hi Nitin:
Thanks for replying.
See my reply.
On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Ni...@citrix.com> wrote:
>
> Lu,
>
> Can you paste the snippet of logs where you see the error? Can you please elaborate on what you mean by secondary storage setup is not working.
> In my knowledge, MS doesn't validate the secondary storage path and does very basic tests while adding it. Typically I have seen the following issues users commit while adding secondary storage.
> Please check if your issue falls under one of them.
>
> 1. Users skip the preparation of the storage (e.g. not seeding the System VM to the NFS export or seeding the wrong version for the hypervisor)
/usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt -m
/mnt/secondary -u http://download.cloud.com/templates/acton/acton-systemvm-
02062012.vhd.bz2 -h xenserver -F
is proformed in the management server, in which
/data/template/tmpl/1/1 was created, and a vhd file together with a
template.properties was under that folder.
three things I saw:
1. under template section of the cloudstack, I saw
SystemVM Template (XenServer) usdatacenter1 XenServer, and if I open
it, I saw "download complete"
However, seems it have nothing to do with the real storage, as if I
delete the secondary storage, it is still there.
2. in the installation guide it says it tooks up to 30 mins to
download that template, however, I saw "successful installation" after
about 30 seconds.
3. seems cloud stack does not check the secondary storage, i tried to
add random IP address with random path, it still shows "enabled".
>
> 2. NFS v4 is used which causes errors during Secondary Storage preparation (Solution: mount the storage and force NFS v3 before preparing the Storage: mount -v -t nfs -o vers=3 servername:/nfs/share /mnt/secondary)
output from the management server:
Server rpc stats:
calls badcalls badauth badclnt xdrcall
3803 0 0 0 0
Server nfs v3:
null getattr setattr lookup access readlink
3232 85% 12 0% 0 0% 0 0% 5 0% 0 0%
read write create mkdir symlink mknod
545 14% 0 0% 0 0% 0 0% 0 0% 0 0%
remove rmdir rename link readdir readdirplus
0 0% 0 0% 0 0% 0 0% 0 0% 3 0%
fsstat fsinfo pathconf commit
0 0% 4 0% 0 0% 0 0%
> 3. The /etc/export file on the NFS does not contain 'no_root_squash'
It does.
> 4. The /etc/export file on the NFS blocks the host/Management Network CIDR
the NFS is on the management server.
> 5. Users enter wrong NFS share path (no validation by CloudStack)
it is correct as I have successfully mounted it from Host sever.
>
> Thanks,
> -Nitin
>
> -----Original Message-----
> From: Lu Heng [mailto:h.lu@anytimechinese.com]
> Sent: Tuesday, June 12, 2012 7:59 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Secondary storage is not working
>
> Hi
>
> I have secondary storage setup at management server's /data path, and while I add it to the zone, it is not working(with information server:<management-server-IP> path:/data), however,I can successfully mount secondary storage in the HV server, any one have idea what is wrong?
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
RE: Secondary storage is not working
Posted by Nitin Mehta <Ni...@citrix.com>.
Lu,
Can you paste the snippet of logs where you see the error? Can you please elaborate on what you mean by secondary storage setup is not working.
In my knowledge, MS doesn't validate the secondary storage path and does very basic tests while adding it. Typically I have seen the following issues users commit while adding secondary storage.
Please check if your issue falls under one of them.
1. Users skip the preparation of the storage (e.g. not seeding the System VM to the NFS export or seeding the wrong version for the hypervisor)
2. NFS v4 is used which causes errors during Secondary Storage preparation (Solution: mount the storage and force NFS v3 before preparing the Storage: mount -v -t nfs -o vers=3 servername:/nfs/share /mnt/secondary)
3. The /etc/export file on the NFS does not contain 'no_root_squash'
4. The /etc/export file on the NFS blocks the host/Management Network CIDR
5. Users enter wrong NFS share path (no validation by CloudStack)
Thanks,
-Nitin
-----Original Message-----
From: Lu Heng [mailto:h.lu@anytimechinese.com]
Sent: Tuesday, June 12, 2012 7:59 AM
To: cloudstack-dev@incubator.apache.org
Subject: Secondary storage is not working
Hi
I have secondary storage setup at management server's /data path, and while I add it to the zone, it is not working(with information server:<management-server-IP> path:/data), however,I can successfully mount secondary storage in the HV server, any one have idea what is wrong?
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or otherwise protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons other than the intended addressee(s) is strictly prohibited. If you have received this transmission in error, please notify this office immediately and e-mail the original at the sender's address above by replying to this message and including the text of the transmission received.