You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Yiping Zhang <yz...@marketo.com> on 2018/11/15 22:01:14 UTC
primary storage best practices?
Hi, all:
At my site, our currently practice is to have only one primary storage device for each CloudStack cluster, serving up to 500 disk volumes with total of 10 – 20TB disk space. Now, we are replacing old NetApp clusters with new ones and moving to SSD disks, so I need to recreate all my primary storage devices.
I am thinking of configuring three primary storage volumes, each served by a different NetApp cluster, for each CloudStack cluster to divide work load on the NetApp end, and to provide some storage redundancy in CloudStack.
My question is when creating new VM instances, how would I distribute new disk volumes on to different primary storage devices evenly and automatically?
I am wondering how are other users configure their (NFS) primary storage devices? What are your best practices in this area?
Thanks
Yiping
Re: primary storage best practices?
Posted by Andrija Panic <an...@gmail.com>.
@Ivan, I'm assunibng Yiping meant other users of CLoudStack (not users
inside CLoudStack) - so yes for us admins...
So we are talking about deployment planner - in similar way as we have
couple of them for the VM deployment (UserDispersing, UserConcetrated, etc)
I like the idea in general.
On Fri, 16 Nov 2018 at 20:29, Yiping Zhang <yz...@marketo.com> wrote:
> Hi, Ivan:
>
> I think one or more deployment planner for storage to handle automatic
> storage placement for new images is a good idea (when multiple primary
> storages are available). But on top of that, letting admins to manually
> pick storage device (to override deployment planner selection) is also a
> good thing to have, giving that it is simply not possible for any
> deployment planner to handle all possible situations out there.
>
> Yiping
>
> On 11/16/18, 10:49 AM, "Ivan Kudryavtsev" <ku...@bw-sw.com>
> wrote:
>
> Hi, Yiping. This is important feature especially for those, who use
> local
> storage deployments.
>
> But I don't think regular users must be able doing that. Admins may
> have
> that feature, but users must perceipt the cloud as incapsulated service
> with hidden topology. What they need is a deployment planner for a
> storage.
>
> The request itself is useful, but the feature design must fit every
> kind of
> cloud use case, not only yours.
>
>
> пт, 16 нояб. 2018 г., 13:10 Yiping Zhang yzhang@marketo.com:
>
> > It sounds like we have an enhancement/feature request here: to be
> able to
> > specify primary storage device where the new image to be created on
> when
> > calling deployVirtualMachine API.
> >
> > Where should I file this request, in Github or the original Apache's
> > CloudStack Jira?
> >
> > Yiping
> >
> > On 11/15/18, 2:27 PM, "Andrija Panic" <an...@gmail.com>
> wrote:
> >
> > I believe (if not mistaken) that CloudStack will match first
> available
> > storage based on storage tags and availability, and will always
> choose
> > first storage pool, even though you have 3 of them available for
> > particular
> > cluster.
> > In this sense, you can not really balance load across multiple
> Primary
> > Storages... (I have actually just tested this, having 2 pools
> with same
> > storage tag, and deploying a few volumes - all of them were
> created on
> > first storage available...)
> >
> > You could configure them with different storage tags, but not
> sure that
> > solves your problem - i.e. some Compute/Disk offerings will be
> > targeting
> > NetApp Cluster1, some NetApp 2, some NetApp3 - but this is
> impractical.
> >
> > Not sure if someone else can shred some light on this scenario ?
> (I
> > could
> > atm imagine a very specific game with editing storage tags on
> > storage_pool
> > via SQL (scheduled job), in order to "rotate" list of available
> storage
> > pools...)
> >
> > Cheers
> >
> > On Thu, 15 Nov 2018 at 23:01, Yiping Zhang <yz...@marketo.com>
> wrote:
> >
> > > Hi, all:
> > >
> > > At my site, our currently practice is to have only one primary
> > storage
> > > device for each CloudStack cluster, serving up to 500 disk
> volumes
> > with
> > > total of 10 – 20TB disk space. Now, we are replacing old
> NetApp
> > clusters
> > > with new ones and moving to SSD disks, so I need to recreate
> all my
> > > primary storage devices.
> > >
> > > I am thinking of configuring three primary storage volumes,
> each
> > served by
> > > a different NetApp cluster, for each CloudStack cluster to
> divide
> > work
> > > load on the NetApp end, and to provide some storage redundancy
> in
> > > CloudStack.
> > >
> > > My question is when creating new VM instances, how would I
> > distribute new
> > > disk volumes on to different primary storage devices evenly and
> > > automatically?
> > >
> > > I am wondering how are other users configure their (NFS)
> primary
> > storage
> > > devices? What are your best practices in this area?
> > >
> > > Thanks
> > >
> > > Yiping
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
> >
> >
>
>
>
--
Andrija Panić
Re: primary storage best practices?
Posted by Yiping Zhang <yz...@marketo.com>.
Hi, Ivan:
I think one or more deployment planner for storage to handle automatic storage placement for new images is a good idea (when multiple primary storages are available). But on top of that, letting admins to manually pick storage device (to override deployment planner selection) is also a good thing to have, giving that it is simply not possible for any deployment planner to handle all possible situations out there.
Yiping
On 11/16/18, 10:49 AM, "Ivan Kudryavtsev" <ku...@bw-sw.com> wrote:
Hi, Yiping. This is important feature especially for those, who use local
storage deployments.
But I don't think regular users must be able doing that. Admins may have
that feature, but users must perceipt the cloud as incapsulated service
with hidden topology. What they need is a deployment planner for a storage.
The request itself is useful, but the feature design must fit every kind of
cloud use case, not only yours.
пт, 16 нояб. 2018 г., 13:10 Yiping Zhang yzhang@marketo.com:
> It sounds like we have an enhancement/feature request here: to be able to
> specify primary storage device where the new image to be created on when
> calling deployVirtualMachine API.
>
> Where should I file this request, in Github or the original Apache's
> CloudStack Jira?
>
> Yiping
>
> On 11/15/18, 2:27 PM, "Andrija Panic" <an...@gmail.com> wrote:
>
> I believe (if not mistaken) that CloudStack will match first available
> storage based on storage tags and availability, and will always choose
> first storage pool, even though you have 3 of them available for
> particular
> cluster.
> In this sense, you can not really balance load across multiple Primary
> Storages... (I have actually just tested this, having 2 pools with same
> storage tag, and deploying a few volumes - all of them were created on
> first storage available...)
>
> You could configure them with different storage tags, but not sure that
> solves your problem - i.e. some Compute/Disk offerings will be
> targeting
> NetApp Cluster1, some NetApp 2, some NetApp3 - but this is impractical.
>
> Not sure if someone else can shred some light on this scenario ? (I
> could
> atm imagine a very specific game with editing storage tags on
> storage_pool
> via SQL (scheduled job), in order to "rotate" list of available storage
> pools...)
>
> Cheers
>
> On Thu, 15 Nov 2018 at 23:01, Yiping Zhang <yz...@marketo.com> wrote:
>
> > Hi, all:
> >
> > At my site, our currently practice is to have only one primary
> storage
> > device for each CloudStack cluster, serving up to 500 disk volumes
> with
> > total of 10 – 20TB disk space. Now, we are replacing old NetApp
> clusters
> > with new ones and moving to SSD disks, so I need to recreate all my
> > primary storage devices.
> >
> > I am thinking of configuring three primary storage volumes, each
> served by
> > a different NetApp cluster, for each CloudStack cluster to divide
> work
> > load on the NetApp end, and to provide some storage redundancy in
> > CloudStack.
> >
> > My question is when creating new VM instances, how would I
> distribute new
> > disk volumes on to different primary storage devices evenly and
> > automatically?
> >
> > I am wondering how are other users configure their (NFS) primary
> storage
> > devices? What are your best practices in this area?
> >
> > Thanks
> >
> > Yiping
> >
>
>
> --
>
> Andrija Panić
>
>
>
Re: primary storage best practices?
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Hi, Yiping. This is important feature especially for those, who use local
storage deployments.
But I don't think regular users must be able doing that. Admins may have
that feature, but users must perceipt the cloud as incapsulated service
with hidden topology. What they need is a deployment planner for a storage.
The request itself is useful, but the feature design must fit every kind of
cloud use case, not only yours.
пт, 16 нояб. 2018 г., 13:10 Yiping Zhang yzhang@marketo.com:
> It sounds like we have an enhancement/feature request here: to be able to
> specify primary storage device where the new image to be created on when
> calling deployVirtualMachine API.
>
> Where should I file this request, in Github or the original Apache's
> CloudStack Jira?
>
> Yiping
>
> On 11/15/18, 2:27 PM, "Andrija Panic" <an...@gmail.com> wrote:
>
> I believe (if not mistaken) that CloudStack will match first available
> storage based on storage tags and availability, and will always choose
> first storage pool, even though you have 3 of them available for
> particular
> cluster.
> In this sense, you can not really balance load across multiple Primary
> Storages... (I have actually just tested this, having 2 pools with same
> storage tag, and deploying a few volumes - all of them were created on
> first storage available...)
>
> You could configure them with different storage tags, but not sure that
> solves your problem - i.e. some Compute/Disk offerings will be
> targeting
> NetApp Cluster1, some NetApp 2, some NetApp3 - but this is impractical.
>
> Not sure if someone else can shred some light on this scenario ? (I
> could
> atm imagine a very specific game with editing storage tags on
> storage_pool
> via SQL (scheduled job), in order to "rotate" list of available storage
> pools...)
>
> Cheers
>
> On Thu, 15 Nov 2018 at 23:01, Yiping Zhang <yz...@marketo.com> wrote:
>
> > Hi, all:
> >
> > At my site, our currently practice is to have only one primary
> storage
> > device for each CloudStack cluster, serving up to 500 disk volumes
> with
> > total of 10 – 20TB disk space. Now, we are replacing old NetApp
> clusters
> > with new ones and moving to SSD disks, so I need to recreate all my
> > primary storage devices.
> >
> > I am thinking of configuring three primary storage volumes, each
> served by
> > a different NetApp cluster, for each CloudStack cluster to divide
> work
> > load on the NetApp end, and to provide some storage redundancy in
> > CloudStack.
> >
> > My question is when creating new VM instances, how would I
> distribute new
> > disk volumes on to different primary storage devices evenly and
> > automatically?
> >
> > I am wondering how are other users configure their (NFS) primary
> storage
> > devices? What are your best practices in this area?
> >
> > Thanks
> >
> > Yiping
> >
>
>
> --
>
> Andrija Panić
>
>
>
Re: primary storage best practices?
Posted by Andrija Panic <an...@gmail.com>.
I'm *assuming* it should be GitHub, but since I'm not a developer, don't
rely on me... I believe there were plans to move away from Jira to GitHub...
But perhaps someone else can also jump in to confirm the Primary Storage
behavior described above...
On Fri, 16 Nov 2018 at 19:10, Yiping Zhang <yz...@marketo.com> wrote:
> It sounds like we have an enhancement/feature request here: to be able to
> specify primary storage device where the new image to be created on when
> calling deployVirtualMachine API.
>
> Where should I file this request, in Github or the original Apache's
> CloudStack Jira?
>
> Yiping
>
> On 11/15/18, 2:27 PM, "Andrija Panic" <an...@gmail.com> wrote:
>
> I believe (if not mistaken) that CloudStack will match first available
> storage based on storage tags and availability, and will always choose
> first storage pool, even though you have 3 of them available for
> particular
> cluster.
> In this sense, you can not really balance load across multiple Primary
> Storages... (I have actually just tested this, having 2 pools with same
> storage tag, and deploying a few volumes - all of them were created on
> first storage available...)
>
> You could configure them with different storage tags, but not sure that
> solves your problem - i.e. some Compute/Disk offerings will be
> targeting
> NetApp Cluster1, some NetApp 2, some NetApp3 - but this is impractical.
>
> Not sure if someone else can shred some light on this scenario ? (I
> could
> atm imagine a very specific game with editing storage tags on
> storage_pool
> via SQL (scheduled job), in order to "rotate" list of available storage
> pools...)
>
> Cheers
>
> On Thu, 15 Nov 2018 at 23:01, Yiping Zhang <yz...@marketo.com> wrote:
>
> > Hi, all:
> >
> > At my site, our currently practice is to have only one primary
> storage
> > device for each CloudStack cluster, serving up to 500 disk volumes
> with
> > total of 10 – 20TB disk space. Now, we are replacing old NetApp
> clusters
> > with new ones and moving to SSD disks, so I need to recreate all my
> > primary storage devices.
> >
> > I am thinking of configuring three primary storage volumes, each
> served by
> > a different NetApp cluster, for each CloudStack cluster to divide
> work
> > load on the NetApp end, and to provide some storage redundancy in
> > CloudStack.
> >
> > My question is when creating new VM instances, how would I
> distribute new
> > disk volumes on to different primary storage devices evenly and
> > automatically?
> >
> > I am wondering how are other users configure their (NFS) primary
> storage
> > devices? What are your best practices in this area?
> >
> > Thanks
> >
> > Yiping
> >
>
>
> --
>
> Andrija Panić
>
>
>
--
Andrija Panić
Re: primary storage best practices?
Posted by Yiping Zhang <yz...@marketo.com>.
It sounds like we have an enhancement/feature request here: to be able to specify primary storage device where the new image to be created on when calling deployVirtualMachine API.
Where should I file this request, in Github or the original Apache's CloudStack Jira?
Yiping
On 11/15/18, 2:27 PM, "Andrija Panic" <an...@gmail.com> wrote:
I believe (if not mistaken) that CloudStack will match first available
storage based on storage tags and availability, and will always choose
first storage pool, even though you have 3 of them available for particular
cluster.
In this sense, you can not really balance load across multiple Primary
Storages... (I have actually just tested this, having 2 pools with same
storage tag, and deploying a few volumes - all of them were created on
first storage available...)
You could configure them with different storage tags, but not sure that
solves your problem - i.e. some Compute/Disk offerings will be targeting
NetApp Cluster1, some NetApp 2, some NetApp3 - but this is impractical.
Not sure if someone else can shred some light on this scenario ? (I could
atm imagine a very specific game with editing storage tags on storage_pool
via SQL (scheduled job), in order to "rotate" list of available storage
pools...)
Cheers
On Thu, 15 Nov 2018 at 23:01, Yiping Zhang <yz...@marketo.com> wrote:
> Hi, all:
>
> At my site, our currently practice is to have only one primary storage
> device for each CloudStack cluster, serving up to 500 disk volumes with
> total of 10 – 20TB disk space. Now, we are replacing old NetApp clusters
> with new ones and moving to SSD disks, so I need to recreate all my
> primary storage devices.
>
> I am thinking of configuring three primary storage volumes, each served by
> a different NetApp cluster, for each CloudStack cluster to divide work
> load on the NetApp end, and to provide some storage redundancy in
> CloudStack.
>
> My question is when creating new VM instances, how would I distribute new
> disk volumes on to different primary storage devices evenly and
> automatically?
>
> I am wondering how are other users configure their (NFS) primary storage
> devices? What are your best practices in this area?
>
> Thanks
>
> Yiping
>
--
Andrija Panić
Re: primary storage best practices?
Posted by Andrija Panic <an...@gmail.com>.
I believe (if not mistaken) that CloudStack will match first available
storage based on storage tags and availability, and will always choose
first storage pool, even though you have 3 of them available for particular
cluster.
In this sense, you can not really balance load across multiple Primary
Storages... (I have actually just tested this, having 2 pools with same
storage tag, and deploying a few volumes - all of them were created on
first storage available...)
You could configure them with different storage tags, but not sure that
solves your problem - i.e. some Compute/Disk offerings will be targeting
NetApp Cluster1, some NetApp 2, some NetApp3 - but this is impractical.
Not sure if someone else can shred some light on this scenario ? (I could
atm imagine a very specific game with editing storage tags on storage_pool
via SQL (scheduled job), in order to "rotate" list of available storage
pools...)
Cheers
On Thu, 15 Nov 2018 at 23:01, Yiping Zhang <yz...@marketo.com> wrote:
> Hi, all:
>
> At my site, our currently practice is to have only one primary storage
> device for each CloudStack cluster, serving up to 500 disk volumes with
> total of 10 – 20TB disk space. Now, we are replacing old NetApp clusters
> with new ones and moving to SSD disks, so I need to recreate all my
> primary storage devices.
>
> I am thinking of configuring three primary storage volumes, each served by
> a different NetApp cluster, for each CloudStack cluster to divide work
> load on the NetApp end, and to provide some storage redundancy in
> CloudStack.
>
> My question is when creating new VM instances, how would I distribute new
> disk volumes on to different primary storage devices evenly and
> automatically?
>
> I am wondering how are other users configure their (NFS) primary storage
> devices? What are your best practices in this area?
>
> Thanks
>
> Yiping
>
--
Andrija Panić