You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Geoff Higginbottom <ge...@shapeblue.com> on 2012/06/22 13:00:26 UTC
Storage Networks XenServer
Hi
I have been trying to get a system up and running using XenServer 6.0.2 with multiple physical networks, with one nic dedicated to Storage.
There seems to be a lot of contradictory info regarding the use of the storage network, and whether it is for Primary or Secondary storage.
The latest admin guide for 3.0.3 states the following
Storage Network Topology Requirements
The secondary storage NFS export is mounted by the secondary storage VM. Secondary storage traffic goes over the management traffic network, even if there is a separate storage network. Primary storage traffic goes over the storage network, if available. If you choose to place secondary storage NFS servers on the storage network, you must make sure there is a route from the management traffic network to the storage network.
And
Separate Storage Network for XenServer (Optional)
You can optionally set up a separate storage network. This should be done first on the host, before implementing the bonding steps below. This can be done using one or two available NICs. With two NICs bonding may be done as above. It is the administrator's responsibility to set up a separate storage network.
Give the storage network a different name-label than what will be given for other networks.
For the separate storage network to work correctly, it must be the only interface that can ping the primary storage device's IP address. For example, if eth0 is the management network NIC, ping -I eth0 <primary storage device IP> must fail. In all deployments, secondary storage devices must be pingable from the management network NIC or bond. If a secondary storage device has been placed on the storage network, it must also be pingable via the storage network NIC or bond on the hosts as well.
I think it is pretty clear that a "Storage Network" in CloudStack is for Primary Storage, but can optionally be used for Secondary Storage.
So I have a test system with single XenServer 6.0.2 Host, with 4 physical NICs, and each NIC is assigned a role with the appropriate traffic label eg
NIC 0: Management
NIC 1: Guest
NIC 2: Public
NIC 3: Storage
The CS Management Server, Host and Secondary Storage are all on the 192.168.1.0/24 address range with no VLAN tagging
The Primary Storage is on the 172.16.0.0/24 address range with no VLAN tagging
Deploying a new Zone works OK, and the System VMs (SSVM and CP) come on line, however uploading templates or ISOs fails.
When I log onto the SSVM and try and ping the Secondary Storage IP (192.168.1.100) it fails as there is a ROUTE forcing the traffic over the Storage Network NIC which is a completely different physical network rung on IP Range 172.16.0.0. If I manually delete the ROUTE then the SSVM can ping the Secondary Storage and the Templates and ISOs upload correctly.
Can someone clarify how we should be using the Storage Network ideally with some detailed examples including IPs
Thanks
Geoff
ShapeBlue provides a range of strategic and technical consulting and implementation services to help IT Service Providers and Enterprises to build a true IaaS compute cloud. ShapeBlue's expertise, combined with CloudStack technology, allows IT Service Providers and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
________________________________
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales.
RE: Storage Networks XenServer
Posted by Frank Zhang <Fr...@citrix.com>.
Yes it's quite confusing.
My best guess, just guess, is CloudStack used to have similar feature but somehow dropped it at some moment, however, the doc was not updated promptly.
Anyway, the storage network in current code base means secondary storage.
Could you file a bug on http://bugs.cloudstack.org/ for this issue? Thank you.
> -----Original Message-----
> From: Geoff Higginbottom [mailto:geoff.higginbottom@shapeblue.com]
> Sent: Friday, June 22, 2012 12:43 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Storage Networks XenServer
>
> Hi Frank
>
> If the storage network is for Secondary Storage, why does the admin guide
> contain the following
>
> "For the separate storage network to work correctly, it must be the only
> interface that can ping the primary storage device's IP address. For example,
> if eth0 is the management network NIC, ping -I eth0 <primary storage device
> IP> must fail. In all deployments, secondary storage devices must be pingable
> from the management network NIC or bond. If a secondary storage device
> has been placed on the storage network, it must also be pingable via the
> storage network NIC or bond on the hosts as well."
>
> And it also states
>
> "The secondary storage NFS export is mounted by the secondary storage VM.
> Secondary storage traffic goes over the management traffic network, even if
> there is a separate storage network. Primary storage traffic goes over the
> storage network, if available. If you choose to place secondary storage NFS
> servers on the storage network, you must make sure there is a route from
> the management traffic network to the storage network."
>
> This seems to make it very clear that the "Storage Network" is for Primary
> Storage, not Secondary.
>
> Geoff
>
>
> -----Original Message-----
> From: Frank Zhang [mailto:Frank.Zhang@citrix.com]
> Sent: 22 June 2012 19:45
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Storage Networks XenServer
>
> See my another reply.
> The storage network(I admit it's a bad term for this feature) is indeed for
> secondary storage now.
>
> > -----Original Message-----
> > From: Weiran Xia [mailto:weiran.xia1@gmail.com]
> > Sent: Friday, June 22, 2012 4:57 AM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: Re: Storage Networks XenServer
> >
> > here i got a similar question, there seems be some confusion in
> terminology.
> >
> > From
> > http://wiki.cloudstack.org/display/DesignDocs/Storage+network+in+Acton
> > it says a storage network is for speeding up template/snapshot copying
> > process, and indeed during the zone creation process there is a step
> > of configuring this network(ip range,gateway..). when the SSVM is up
> > and running it gets 4 NICs, one of them is dedicated for this purpose.
> >
> > But where is the storage network for primary storage? is this storage
> > network only defined by label in step two (drag and drop) of zone creation?
> > is there any relationship between two 'storage networks'?
> >
> > thanks very much if someone can clarify the storage nework in 3.0.x
> >
> > Regards
> > Mice
>
> ShapeBlue provides a range of strategic and technical consulting and
> implementation services to help IT Service Providers and Enterprises to build
> a true IaaS compute cloud. ShapeBlue’s expertise, combined with CloudStack
> technology, allows IT Service Providers and Enterprises to deliver true, utility
> based, IaaS to the customer or end-user.
>
> ________________________________
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd. If you are not the intended recipient of
> this email, you must neither take any action based upon its contents, nor
> copy or show it to anyone. Please contact the sender if you believe you have
> received this email in error. Shape Blue Ltd is a company incorporated in
> England & Wales.
RE: Storage Networks XenServer
Posted by Geoff Higginbottom <ge...@shapeblue.com>.
Hi Frank
If the storage network is for Secondary Storage, why does the admin guide contain the following
"For the separate storage network to work correctly, it must be the only interface that can ping the primary storage device's IP address. For example, if eth0 is the management network NIC, ping -I eth0 <primary storage device IP> must fail. In all deployments, secondary storage devices must be pingable from the management network NIC or bond. If a secondary storage device has been placed on the storage network, it must also be pingable via the storage network NIC or bond on the hosts as well."
And it also states
"The secondary storage NFS export is mounted by the secondary storage VM. Secondary storage traffic goes over the management traffic network, even if there is a separate storage network. Primary storage traffic goes over the storage network, if available. If you choose to place secondary storage NFS servers on the storage network, you must make sure there is a route from the management traffic network to the storage network."
This seems to make it very clear that the "Storage Network" is for Primary Storage, not Secondary.
Geoff
-----Original Message-----
From: Frank Zhang [mailto:Frank.Zhang@citrix.com]
Sent: 22 June 2012 19:45
To: cloudstack-dev@incubator.apache.org
Subject: RE: Storage Networks XenServer
See my another reply.
The storage network(I admit it's a bad term for this feature) is indeed for secondary storage now.
> -----Original Message-----
> From: Weiran Xia [mailto:weiran.xia1@gmail.com]
> Sent: Friday, June 22, 2012 4:57 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Storage Networks XenServer
>
> here i got a similar question, there seems be some confusion in terminology.
>
> From
> http://wiki.cloudstack.org/display/DesignDocs/Storage+network+in+Acton
> it says a storage network is for speeding up template/snapshot copying
> process, and indeed during the zone creation process there is a step
> of configuring this network(ip range,gateway..). when the SSVM is up
> and running it gets 4 NICs, one of them is dedicated for this purpose.
>
> But where is the storage network for primary storage? is this storage
> network only defined by label in step two (drag and drop) of zone creation?
> is there any relationship between two 'storage networks'?
>
> thanks very much if someone can clarify the storage nework in 3.0.x
>
> Regards
> Mice
ShapeBlue provides a range of strategic and technical consulting and implementation services to help IT Service Providers and Enterprises to build a true IaaS compute cloud. ShapeBlue’s expertise, combined with CloudStack technology, allows IT Service Providers and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
________________________________
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales.
RE: Storage Networks XenServer
Posted by Frank Zhang <Fr...@citrix.com>.
See my another reply.
The storage network(I admit it's a bad term for this feature) is indeed for secondary storage now.
> -----Original Message-----
> From: Weiran Xia [mailto:weiran.xia1@gmail.com]
> Sent: Friday, June 22, 2012 4:57 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Storage Networks XenServer
>
> here i got a similar question, there seems be some confusion in terminology.
>
> From
> http://wiki.cloudstack.org/display/DesignDocs/Storage+network+in+Acton
> it says a storage network is for speeding up template/snapshot copying
> process, and indeed during the zone creation process there is a step of
> configuring this network(ip range,gateway..). when the SSVM is up and
> running it gets 4 NICs, one of them is dedicated for this purpose.
>
> But where is the storage network for primary storage? is this storage
> network only defined by label in step two (drag and drop) of zone creation?
> is there any relationship between two 'storage networks'?
>
> thanks very much if someone can clarify the storage nework in 3.0.x
>
> Regards
> Mice
Re: Storage Networks XenServer
Posted by Weiran Xia <we...@gmail.com>.
here i got a similar question, there seems be some confusion in terminology.
>From http://wiki.cloudstack.org/display/DesignDocs/Storage+network+in+Acton
it says a storage network is for speeding up template/snapshot copying
process, and indeed during the zone creation process there is a step of
configuring this network(ip range,gateway..). when the SSVM is up and
running it gets 4 NICs, one of them is dedicated for this purpose.
But where is the storage network for primary storage? is this storage
network only defined by label in step two (drag and drop) of zone creation?
is there any relationship between two 'storage networks'?
thanks very much if someone can clarify the storage nework in 3.0.x
Regards
Mice
Re: Storage Networks XenServer
Posted by Chip Childers <ch...@sungard.com>.
Frank,
One last followup:
On Fri, Jun 22, 2012 at 4:48 PM, Frank Zhang <Fr...@citrix.com> wrote:
> Let's say the host having two nics, one is staying on a high bandwidth
> network that can access primary storage(the nfs share), another is on a
> different network.
> Ideally a route specifying that access to primary storage should go thru
> high bandwidth network will make mount perform faster. But CloudStack
> hasn't set such sort of route on host for primary storage yet.
In the scenario where the primary storage target IP is within the same IP
block as the "high bandwidth" interface you describe, I don't think that a
route should be needed beyond the normal block level route that sends
traffic out that interface. My assumption is that the host's default
gateway is going out the other interface in this scenario.
Do you agree that this is the case, and that the issue only pertains to
scenarios where the traffic would need a specific route to send traffic to
a gateway on that storage network?
-chip
RE: Storage Networks XenServer
Posted by Frank Zhang <Fr...@citrix.com>.
Let's say the host having two nics, one is staying on a high bandwidth network that can access primary storage(the nfs share), another is on a different network.
Ideally a route specifying that access to primary storage should go thru high bandwidth network will make mount perform faster. But CloudStack hasn't set such sort of route on host for primary storage yet.
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Friday, June 22, 2012 11:48 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Storage Networks XenServer
>
> Hi Frank... can you clarify this statement a little more for me please?
>
> On Fri, Jun 22, 2012 at 2:43 PM, Frank Zhang <Fr...@citrix.com> wrote:
>
> > As far as I know, CloudStack hasn't been capable of leveraging
> > dedicated nic on host for primary storage.
>
>
> Thanks a ton!
Re: Storage Networks XenServer
Posted by Chip Childers <ch...@sungard.com>.
Hi Frank... can you clarify this statement a little more for me please?
On Fri, Jun 22, 2012 at 2:43 PM, Frank Zhang <Fr...@citrix.com> wrote:
> As far as I know, CloudStack hasn't been capable of leveraging dedicated
> nic on host for primary storage.
Thanks a ton!
RE: Storage Networks XenServer
Posted by Frank Zhang <Fr...@citrix.com>.
I apologize so many puzzling terminologies here, to your surprise, the storage network in CloudStack is currently for secondary storage.
http://wiki.cloudstack.org/display/DesignDocs/Storage+network+in+Acton has a short explanation.
To people's intuition, I quite understand the storage network should imply primary storage, we used confusing notion in this part.
As far as I know, CloudStack hasn't been capable of leveraging dedicated nic on host for primary storage.
> -----Original Message-----
> From: Geoff Higginbottom [mailto:geoff.higginbottom@shapeblue.com]
> Sent: Friday, June 22, 2012 4:00 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Storage Networks XenServer
>
> Hi
>
> I have been trying to get a system up and running using XenServer 6.0.2 with
> multiple physical networks, with one nic dedicated to Storage.
>
> There seems to be a lot of contradictory info regarding the use of the storage
> network, and whether it is for Primary or Secondary storage.
>
> The latest admin guide for 3.0.3 states the following
>
>
> Storage Network Topology Requirements
> The secondary storage NFS export is mounted by the secondary storage VM.
> Secondary storage traffic goes over the management traffic network, even if
> there is a separate storage network. Primary storage traffic goes over the
> storage network, if available. If you choose to place secondary storage NFS
> servers on the storage network, you must make sure there is a route from
> the management traffic network to the storage network.
>
> And
>
>
> Separate Storage Network for XenServer (Optional)
>
> You can optionally set up a separate storage network. This should be done
> first on the host, before implementing the bonding steps below. This can be
> done using one or two available NICs. With two NICs bonding may be done as
> above. It is the administrator's responsibility to set up a separate storage
> network.
>
>
>
> Give the storage network a different name-label than what will be given for
> other networks.
>
> For the separate storage network to work correctly, it must be the only
> interface that can ping the primary storage device's IP address. For example,
> if eth0 is the management network NIC, ping -I eth0 <primary storage device
> IP> must fail. In all deployments, secondary storage devices must be pingable
> from the management network NIC or bond. If a secondary storage device
> has been placed on the storage network, it must also be pingable via the
> storage network NIC or bond on the hosts as well.
>
>
> I think it is pretty clear that a "Storage Network" in CloudStack is for Primary
> Storage, but can optionally be used for Secondary Storage.
>
> So I have a test system with single XenServer 6.0.2 Host, with 4 physical NICs,
> and each NIC is assigned a role with the appropriate traffic label eg NIC 0:
> Management NIC 1: Guest NIC 2: Public NIC 3: Storage
>
> The CS Management Server, Host and Secondary Storage are all on the
> 192.168.1.0/24 address range with no VLAN tagging The Primary Storage is on
> the 172.16.0.0/24 address range with no VLAN tagging
>
> Deploying a new Zone works OK, and the System VMs (SSVM and CP) come
> on line, however uploading templates or ISOs fails.
>
> When I log onto the SSVM and try and ping the Secondary Storage IP
> (192.168.1.100) it fails as there is a ROUTE forcing the traffic over the Storage
> Network NIC which is a completely different physical network rung on IP
> Range 172.16.0.0. If I manually delete the ROUTE then the SSVM can ping
> the Secondary Storage and the Templates and ISOs upload correctly.
>
> Can someone clarify how we should be using the Storage Network ideally
> with some detailed examples including IPs
>
> Thanks
>
> Geoff
> ShapeBlue provides a range of strategic and technical consulting and
> implementation services to help IT Service Providers and Enterprises to build
> a true IaaS compute cloud. ShapeBlue's expertise, combined with CloudStack
> technology, allows IT Service Providers and Enterprises to deliver true, utility
> based, IaaS to the customer or end-user.
>
> ________________________________
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd. If you are not the intended recipient of
> this email, you must neither take any action based upon its contents, nor
> copy or show it to anyone. Please contact the sender if you believe you have
> received this email in error. Shape Blue Ltd is a company incorporated in
> England & Wales.