You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Mike Tutkowski <mi...@solidfire.com> on 2013/05/31 06:08:13 UTC
[PROPOSAL] Storage plug-in for 4.2
Hi everyone,
I apologize for being unfamiliar with how I should have gone about getting
consensus for the storage plug-in I developed for 4.2.
I talked with Animesh and he has asked me to send out this proposal related
to the storage plug-in I built for 4.2 and submitted to Review Board
earlier this week.
Here is a link to the design document:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
Here is a link to the JIRA ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-2778
Here is a link to it on Review Board:
https://reviews.apache.org/r/11479/
Here is a high-level summary:
I have developed a storage plug-in which makes use of the new storage
framework that Edison put in place for the 4.2 release.
Working with Edison, I have identified a few areas of the storage framework
that needed to be enhanced (and I have done so in the code that I
submitted) for dynamic, zone-wide primary storage to function.
This storage plug-in is specific to SolidFire (a data-storage company),
whose architecture is designed around Quality of Service. Each volume on
this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
one might expect, a guaranteed number (even in fault conditions) (as long
as the IOPS of the SAN are not over-provisioned).
Per a discussion several months ago on this list, I have added into the
Disk Offerings Min, Max, and Burst values (all optional). These values
follow the patterns of the Disk Size field in that they can be customized
by the end user (in the Add Volume) if the admin decides to allow this.
In the end, this feature allows users to create a 1:1 mapping between a
volume on the SAN and a data disk that can be attached to a VM (XenServer
supported...perhaps VMware, depending on time). This allows CloudStack
admins to offer their end users guaranteed IOPS on data disks (eliminating
the Noisy Neighbor effect). The plan is to support root disks in a post-4.2
release.
Again, I am sorry about my confusion regarding process here. I certainly
appreciate all of the input I have received on the e-mail list over the
past couple months while I was developing this feature.
Please let me know if you have any questions.
Thanks!
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Mike Tutkowski <mi...@solidfire.com>.
No problem! :)
On Fri, May 31, 2013 at 2:01 PM, Wei ZHOU <us...@gmail.com> wrote:
> Mike,
> Thanks a lot!
>
> -Wei
>
> 2013/5/31, Mike Tutkowski <mi...@solidfire.com>:
> > Hi Wei,
> >
> > Thanks!
> >
> > Here are the details you requested. This also talks about Min and Max
> IOPS.
> >
> > Please let me know if you'd like more information. I have an entire
> > document I could share with you regarding SolidFire QoS (the info below
> is
> > taken from this document).
> >
> > Talk to you later!
> >
> > How does SolidFire QoS use burstIOPS?
> >
> > When the cluster is running in a state of low cluster utilization,
> clients
> > are able to accrue burstIOPS credits and use these credits to burst above
> > their max up to their burst IOPS for the burst period.
> >
> > The current burst period is 60 seconds.
> >
> > Burst IOPS credits are accrued by performing less IO than the maxIOPS.
> Each
> > IO that is not used under the maxIOPS, is considered an IOPS “credit”,
> that
> > can be used later.
> >
> > There are two limits to the way that the burst IOPS credits can be used.
> >
> > The first limit is that the total number of burst credits that a customer
> > can accrue is (60 seconds) * (burst IOPS - max IOPS).
> >
> > The second limit is that the number of credits that can be depleted at
> any
> > time is at most the burst IOPS. Thus, a customer will never be able to go
> > higher than the burst IOPS set for the volume.
> >
> > Note: when another volume on the node is unable to reach its minIOPS,
> > burstIOPS are reset for all customers on the node.
> >
> >
> > Basic explanation: When there is performance left to do so, SolidFIre
> > allows customers to burst above their maximum for 60 seconds.
> >
> > How does SolidFire QoS use maxIOPS?
> >
> > MaxIOPS is a sustained rate limit. For a sustained period of time, the
> > customer will be able to sustain this IO rate, as long as no one else on
> > the cluster is unable to get their minimum IOPS.
> >
> > Basic explanation: MaxIOPS is what a customer will sustain for a long
> > period of time if everyone is getting their minimum IOPS guarantees.
> >
> > How does SolidFire QoS use minIOPS?
> >
> > The minIOPS is a lower limit on the performance that a customer should
> > expect in normal operating scenarios.
> >
> > SolidFire QoS is constantly monitoring the latencies observed by the
> > client, and the queue depth that the client is driving to the cluster.
> >
> > When a customer reaches the “cannot reach minIOPS situation”, as detected
> > by the targetIOPS and the latencies that the cluster is seeing at the
> given
> > IOPS, other customers are gradually ratcheted and pushed down to their
> > minIOPS.
> >
> > Note: customers are never throttled to less than their minIOPS unless
> there
> > is serious overload.
> >
> > What this means, is that at sufficiently high queue depth on a heavily
> > utilized cluster, our quality of service detects situations where a
> > customer is unable to get their minIOPS, and takes performance away from
> > other customers running above their minIOPS, and gives it to the customer
> > that is unable to reach their minIOPS until the “unhappy” customer
> reaches
> > their minIOPS.
> >
> > Note: In Boron (Version 5), to qualify for the minIOPS guarantee, the
> > customer must be running a queue depth higher than 8.
> >
> > What these three settings give us is something unique to storage: a
> quality
> > of service that limits the customer’s sustained effect on performance on
> > the cluster (max IOPS), the ability to burst and use unused performance
> > (burst IOPS) if there is any, and a reasonable expectation of performance
> > even in high utilization situations where performance can be dynamically
> > balanced throughout customers in the system.
> >
> >
> > The net effect of this, is that, in SolidFire, because the ratio of
> > performance between customers is changing depending on cluster
> utilization,
> > we’ve made it so that each customer’s performance isn’t directly tied to
> > each other.
> >
> >
> >
> > On Fri, May 31, 2013 at 10:43 AM, Wei ZHOU <us...@gmail.com>
> wrote:
> >
> >> Mike,
> >>
> >> good work.
> >> I notice the burst IOPS. Do you know the mechanism and the config of
> >> it, like the duration and interval burst IO? Thanks.
> >>
> >> -Wei
> >>
> >> 2013/5/31, Mike Tutkowski <mi...@solidfire.com>:
> >> > Hi everyone,
> >> >
> >> > I apologize for being unfamiliar with how I should have gone about
> >> getting
> >> > consensus for the storage plug-in I developed for 4.2.
> >> >
> >> > I talked with Animesh and he has asked me to send out this proposal
> >> related
> >> > to the storage plug-in I built for 4.2 and submitted to Review Board
> >> > earlier this week.
> >> >
> >> > Here is a link to the design document:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
> >> >
> >> > Here is a link to the JIRA ticket:
> >> >
> >> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
> >> >
> >> > Here is a link to it on Review Board:
> >> >
> >> > https://reviews.apache.org/r/11479/
> >> >
> >> > Here is a high-level summary:
> >> >
> >> > I have developed a storage plug-in which makes use of the new storage
> >> > framework that Edison put in place for the 4.2 release.
> >> >
> >> > Working with Edison, I have identified a few areas of the storage
> >> framework
> >> > that needed to be enhanced (and I have done so in the code that I
> >> > submitted) for dynamic, zone-wide primary storage to function.
> >> >
> >> > This storage plug-in is specific to SolidFire (a data-storage
> company),
> >> > whose architecture is designed around Quality of Service. Each volume
> >> > on
> >> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is,
> >> > as
> >> > one might expect, a guaranteed number (even in fault conditions) (as
> >> > long
> >> > as the IOPS of the SAN are not over-provisioned).
> >> >
> >> > Per a discussion several months ago on this list, I have added into
> the
> >> > Disk Offerings Min, Max, and Burst values (all optional). These values
> >> > follow the patterns of the Disk Size field in that they can be
> >> > customized
> >> > by the end user (in the Add Volume) if the admin decides to allow
> this.
> >> >
> >> > In the end, this feature allows users to create a 1:1 mapping between
> a
> >> > volume on the SAN and a data disk that can be attached to a VM
> >> > (XenServer
> >> > supported...perhaps VMware, depending on time). This allows CloudStack
> >> > admins to offer their end users guaranteed IOPS on data disks
> >> (eliminating
> >> > the Noisy Neighbor effect). The plan is to support root disks in a
> >> post-4.2
> >> > release.
> >> >
> >> > Again, I am sorry about my confusion regarding process here. I
> >> > certainly
> >> > appreciate all of the input I have received on the e-mail list over
> the
> >> > past couple months while I was developing this feature.
> >> >
> >> > Please let me know if you have any questions.
> >> >
> >> > Thanks!
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *™*
> >> >
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Wei ZHOU <us...@gmail.com>.
Mike,
Thanks a lot!
-Wei
2013/5/31, Mike Tutkowski <mi...@solidfire.com>:
> Hi Wei,
>
> Thanks!
>
> Here are the details you requested. This also talks about Min and Max IOPS.
>
> Please let me know if you'd like more information. I have an entire
> document I could share with you regarding SolidFire QoS (the info below is
> taken from this document).
>
> Talk to you later!
>
> How does SolidFire QoS use burstIOPS?
>
> When the cluster is running in a state of low cluster utilization, clients
> are able to accrue burstIOPS credits and use these credits to burst above
> their max up to their burst IOPS for the burst period.
>
> The current burst period is 60 seconds.
>
> Burst IOPS credits are accrued by performing less IO than the maxIOPS. Each
> IO that is not used under the maxIOPS, is considered an IOPS “credit”, that
> can be used later.
>
> There are two limits to the way that the burst IOPS credits can be used.
>
> The first limit is that the total number of burst credits that a customer
> can accrue is (60 seconds) * (burst IOPS - max IOPS).
>
> The second limit is that the number of credits that can be depleted at any
> time is at most the burst IOPS. Thus, a customer will never be able to go
> higher than the burst IOPS set for the volume.
>
> Note: when another volume on the node is unable to reach its minIOPS,
> burstIOPS are reset for all customers on the node.
>
>
> Basic explanation: When there is performance left to do so, SolidFIre
> allows customers to burst above their maximum for 60 seconds.
>
> How does SolidFire QoS use maxIOPS?
>
> MaxIOPS is a sustained rate limit. For a sustained period of time, the
> customer will be able to sustain this IO rate, as long as no one else on
> the cluster is unable to get their minimum IOPS.
>
> Basic explanation: MaxIOPS is what a customer will sustain for a long
> period of time if everyone is getting their minimum IOPS guarantees.
>
> How does SolidFire QoS use minIOPS?
>
> The minIOPS is a lower limit on the performance that a customer should
> expect in normal operating scenarios.
>
> SolidFire QoS is constantly monitoring the latencies observed by the
> client, and the queue depth that the client is driving to the cluster.
>
> When a customer reaches the “cannot reach minIOPS situation”, as detected
> by the targetIOPS and the latencies that the cluster is seeing at the given
> IOPS, other customers are gradually ratcheted and pushed down to their
> minIOPS.
>
> Note: customers are never throttled to less than their minIOPS unless there
> is serious overload.
>
> What this means, is that at sufficiently high queue depth on a heavily
> utilized cluster, our quality of service detects situations where a
> customer is unable to get their minIOPS, and takes performance away from
> other customers running above their minIOPS, and gives it to the customer
> that is unable to reach their minIOPS until the “unhappy” customer reaches
> their minIOPS.
>
> Note: In Boron (Version 5), to qualify for the minIOPS guarantee, the
> customer must be running a queue depth higher than 8.
>
> What these three settings give us is something unique to storage: a quality
> of service that limits the customer’s sustained effect on performance on
> the cluster (max IOPS), the ability to burst and use unused performance
> (burst IOPS) if there is any, and a reasonable expectation of performance
> even in high utilization situations where performance can be dynamically
> balanced throughout customers in the system.
>
>
> The net effect of this, is that, in SolidFire, because the ratio of
> performance between customers is changing depending on cluster utilization,
> we’ve made it so that each customer’s performance isn’t directly tied to
> each other.
>
>
>
> On Fri, May 31, 2013 at 10:43 AM, Wei ZHOU <us...@gmail.com> wrote:
>
>> Mike,
>>
>> good work.
>> I notice the burst IOPS. Do you know the mechanism and the config of
>> it, like the duration and interval burst IO? Thanks.
>>
>> -Wei
>>
>> 2013/5/31, Mike Tutkowski <mi...@solidfire.com>:
>> > Hi everyone,
>> >
>> > I apologize for being unfamiliar with how I should have gone about
>> getting
>> > consensus for the storage plug-in I developed for 4.2.
>> >
>> > I talked with Animesh and he has asked me to send out this proposal
>> related
>> > to the storage plug-in I built for 4.2 and submitted to Review Board
>> > earlier this week.
>> >
>> > Here is a link to the design document:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
>> >
>> > Here is a link to the JIRA ticket:
>> >
>> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
>> >
>> > Here is a link to it on Review Board:
>> >
>> > https://reviews.apache.org/r/11479/
>> >
>> > Here is a high-level summary:
>> >
>> > I have developed a storage plug-in which makes use of the new storage
>> > framework that Edison put in place for the 4.2 release.
>> >
>> > Working with Edison, I have identified a few areas of the storage
>> framework
>> > that needed to be enhanced (and I have done so in the code that I
>> > submitted) for dynamic, zone-wide primary storage to function.
>> >
>> > This storage plug-in is specific to SolidFire (a data-storage company),
>> > whose architecture is designed around Quality of Service. Each volume
>> > on
>> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is,
>> > as
>> > one might expect, a guaranteed number (even in fault conditions) (as
>> > long
>> > as the IOPS of the SAN are not over-provisioned).
>> >
>> > Per a discussion several months ago on this list, I have added into the
>> > Disk Offerings Min, Max, and Burst values (all optional). These values
>> > follow the patterns of the Disk Size field in that they can be
>> > customized
>> > by the end user (in the Add Volume) if the admin decides to allow this.
>> >
>> > In the end, this feature allows users to create a 1:1 mapping between a
>> > volume on the SAN and a data disk that can be attached to a VM
>> > (XenServer
>> > supported...perhaps VMware, depending on time). This allows CloudStack
>> > admins to offer their end users guaranteed IOPS on data disks
>> (eliminating
>> > the Noisy Neighbor effect). The plan is to support root disks in a
>> post-4.2
>> > release.
>> >
>> > Again, I am sorry about my confusion regarding process here. I
>> > certainly
>> > appreciate all of the input I have received on the e-mail list over the
>> > past couple months while I was developing this feature.
>> >
>> > Please let me know if you have any questions.
>> >
>> > Thanks!
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>> >
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Wei,
Thanks!
Here are the details you requested. This also talks about Min and Max IOPS.
Please let me know if you'd like more information. I have an entire
document I could share with you regarding SolidFire QoS (the info below is
taken from this document).
Talk to you later!
How does SolidFire QoS use burstIOPS?
When the cluster is running in a state of low cluster utilization, clients
are able to accrue burstIOPS credits and use these credits to burst above
their max up to their burst IOPS for the burst period.
The current burst period is 60 seconds.
Burst IOPS credits are accrued by performing less IO than the maxIOPS. Each
IO that is not used under the maxIOPS, is considered an IOPS “credit”, that
can be used later.
There are two limits to the way that the burst IOPS credits can be used.
The first limit is that the total number of burst credits that a customer
can accrue is (60 seconds) * (burst IOPS - max IOPS).
The second limit is that the number of credits that can be depleted at any
time is at most the burst IOPS. Thus, a customer will never be able to go
higher than the burst IOPS set for the volume.
Note: when another volume on the node is unable to reach its minIOPS,
burstIOPS are reset for all customers on the node.
Basic explanation: When there is performance left to do so, SolidFIre
allows customers to burst above their maximum for 60 seconds.
How does SolidFire QoS use maxIOPS?
MaxIOPS is a sustained rate limit. For a sustained period of time, the
customer will be able to sustain this IO rate, as long as no one else on
the cluster is unable to get their minimum IOPS.
Basic explanation: MaxIOPS is what a customer will sustain for a long
period of time if everyone is getting their minimum IOPS guarantees.
How does SolidFire QoS use minIOPS?
The minIOPS is a lower limit on the performance that a customer should
expect in normal operating scenarios.
SolidFire QoS is constantly monitoring the latencies observed by the
client, and the queue depth that the client is driving to the cluster.
When a customer reaches the “cannot reach minIOPS situation”, as detected
by the targetIOPS and the latencies that the cluster is seeing at the given
IOPS, other customers are gradually ratcheted and pushed down to their
minIOPS.
Note: customers are never throttled to less than their minIOPS unless there
is serious overload.
What this means, is that at sufficiently high queue depth on a heavily
utilized cluster, our quality of service detects situations where a
customer is unable to get their minIOPS, and takes performance away from
other customers running above their minIOPS, and gives it to the customer
that is unable to reach their minIOPS until the “unhappy” customer reaches
their minIOPS.
Note: In Boron (Version 5), to qualify for the minIOPS guarantee, the
customer must be running a queue depth higher than 8.
What these three settings give us is something unique to storage: a quality
of service that limits the customer’s sustained effect on performance on
the cluster (max IOPS), the ability to burst and use unused performance
(burst IOPS) if there is any, and a reasonable expectation of performance
even in high utilization situations where performance can be dynamically
balanced throughout customers in the system.
The net effect of this, is that, in SolidFire, because the ratio of
performance between customers is changing depending on cluster utilization,
we’ve made it so that each customer’s performance isn’t directly tied to
each other.
On Fri, May 31, 2013 at 10:43 AM, Wei ZHOU <us...@gmail.com> wrote:
> Mike,
>
> good work.
> I notice the burst IOPS. Do you know the mechanism and the config of
> it, like the duration and interval burst IO? Thanks.
>
> -Wei
>
> 2013/5/31, Mike Tutkowski <mi...@solidfire.com>:
> > Hi everyone,
> >
> > I apologize for being unfamiliar with how I should have gone about
> getting
> > consensus for the storage plug-in I developed for 4.2.
> >
> > I talked with Animesh and he has asked me to send out this proposal
> related
> > to the storage plug-in I built for 4.2 and submitted to Review Board
> > earlier this week.
> >
> > Here is a link to the design document:
> >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
> >
> > Here is a link to the JIRA ticket:
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
> >
> > Here is a link to it on Review Board:
> >
> > https://reviews.apache.org/r/11479/
> >
> > Here is a high-level summary:
> >
> > I have developed a storage plug-in which makes use of the new storage
> > framework that Edison put in place for the 4.2 release.
> >
> > Working with Edison, I have identified a few areas of the storage
> framework
> > that needed to be enhanced (and I have done so in the code that I
> > submitted) for dynamic, zone-wide primary storage to function.
> >
> > This storage plug-in is specific to SolidFire (a data-storage company),
> > whose architecture is designed around Quality of Service. Each volume on
> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
> > one might expect, a guaranteed number (even in fault conditions) (as long
> > as the IOPS of the SAN are not over-provisioned).
> >
> > Per a discussion several months ago on this list, I have added into the
> > Disk Offerings Min, Max, and Burst values (all optional). These values
> > follow the patterns of the Disk Size field in that they can be customized
> > by the end user (in the Add Volume) if the admin decides to allow this.
> >
> > In the end, this feature allows users to create a 1:1 mapping between a
> > volume on the SAN and a data disk that can be attached to a VM (XenServer
> > supported...perhaps VMware, depending on time). This allows CloudStack
> > admins to offer their end users guaranteed IOPS on data disks
> (eliminating
> > the Noisy Neighbor effect). The plan is to support root disks in a
> post-4.2
> > release.
> >
> > Again, I am sorry about my confusion regarding process here. I certainly
> > appreciate all of the input I have received on the e-mail list over the
> > past couple months while I was developing this feature.
> >
> > Please let me know if you have any questions.
> >
> > Thanks!
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Wei ZHOU <us...@gmail.com>.
Mike,
good work.
I notice the burst IOPS. Do you know the mechanism and the config of
it, like the duration and interval burst IO? Thanks.
-Wei
2013/5/31, Mike Tutkowski <mi...@solidfire.com>:
> Hi everyone,
>
> I apologize for being unfamiliar with how I should have gone about getting
> consensus for the storage plug-in I developed for 4.2.
>
> I talked with Animesh and he has asked me to send out this proposal related
> to the storage plug-in I built for 4.2 and submitted to Review Board
> earlier this week.
>
> Here is a link to the design document:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
>
> Here is a link to the JIRA ticket:
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-2778
>
> Here is a link to it on Review Board:
>
> https://reviews.apache.org/r/11479/
>
> Here is a high-level summary:
>
> I have developed a storage plug-in which makes use of the new storage
> framework that Edison put in place for the 4.2 release.
>
> Working with Edison, I have identified a few areas of the storage framework
> that needed to be enhanced (and I have done so in the code that I
> submitted) for dynamic, zone-wide primary storage to function.
>
> This storage plug-in is specific to SolidFire (a data-storage company),
> whose architecture is designed around Quality of Service. Each volume on
> this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
> one might expect, a guaranteed number (even in fault conditions) (as long
> as the IOPS of the SAN are not over-provisioned).
>
> Per a discussion several months ago on this list, I have added into the
> Disk Offerings Min, Max, and Burst values (all optional). These values
> follow the patterns of the Disk Size field in that they can be customized
> by the end user (in the Add Volume) if the admin decides to allow this.
>
> In the end, this feature allows users to create a 1:1 mapping between a
> volume on the SAN and a data disk that can be attached to a VM (XenServer
> supported...perhaps VMware, depending on time). This allows CloudStack
> admins to offer their end users guaranteed IOPS on data disks (eliminating
> the Noisy Neighbor effect). The plan is to support root disks in a post-4.2
> release.
>
> Again, I am sorry about my confusion regarding process here. I certainly
> appreciate all of the input I have received on the e-mail list over the
> past couple months while I was developing this feature.
>
> Please let me know if you have any questions.
>
> Thanks!
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by John Burwell <jb...@basho.com>.
Mike,
Excellent.
Thanks,
-John
On May 31, 2013, at 10:52 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> I think this is the link you'll need:
>
> https://reviews.apache.org/r/11479/
>
> Thanks! :)
>
>
> On Fri, May 31, 2013 at 8:45 AM, John Burwell <jb...@basho.com> wrote:
> Mike/Animesh,
>
> I am happy to review. However, other $dayjob duties may prevent me from getting to it until Monday. Has the patch been pushed into Review Board?
>
> Thanks,
> -John
>
> On May 31, 2013, at 10:34 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
>> Hi Animesh,
>>
>> Sure, I'd be happy to do that.
>>
>> Would you like these comments in this e-mail chain, the design doc, and/or the JIRA ticket?
>>
>> Also, just to make sure I understand correctly when we refer to "dependencies" here, the plug-in simply requires the storage-framework changes that Edison put in place for 4.2. As long as those are in place (along with the enhancements I made to his storage framework), then the plug-in will function. There are no additional libraries used by the plug-in that were not already in use within CloudStack.
>>
>> Thanks!
>>
>>
>> On Thu, May 30, 2013 at 10:31 PM, Animesh Chaturvedi <an...@citrix.com> wrote:
>> Mike don't feel bad it's my fault I think everyone knew you were working on the plugin actively and I assumed the proposal was already in place.
>>
>> Can you call out your dependencies on Edison's framework?
>>
>> John I added as you as reviewers for Mike's contribution, can you pass on your comments.
>>
>> Thanks
>> Animesh
>>
>> On May 30, 2013, at 9:09 PM, "Mike Tutkowski" <mi...@solidfire.com> wrote:
>>
>> > Hi everyone,
>> >
>> > I apologize for being unfamiliar with how I should have gone about getting
>> > consensus for the storage plug-in I developed for 4.2.
>> >
>> > I talked with Animesh and he has asked me to send out this proposal related
>> > to the storage plug-in I built for 4.2 and submitted to Review Board
>> > earlier this week.
>> >
>> > Here is a link to the design document:
>> >
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
>> >
>> > Here is a link to the JIRA ticket:
>> >
>> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
>> >
>> > Here is a link to it on Review Board:
>> >
>> > https://reviews.apache.org/r/11479/
>> >
>> > Here is a high-level summary:
>> >
>> > I have developed a storage plug-in which makes use of the new storage
>> > framework that Edison put in place for the 4.2 release.
>> >
>> > Working with Edison, I have identified a few areas of the storage framework
>> > that needed to be enhanced (and I have done so in the code that I
>> > submitted) for dynamic, zone-wide primary storage to function.
>> >
>> > This storage plug-in is specific to SolidFire (a data-storage company),
>> > whose architecture is designed around Quality of Service. Each volume on
>> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
>> > one might expect, a guaranteed number (even in fault conditions) (as long
>> > as the IOPS of the SAN are not over-provisioned).
>> >
>> > Per a discussion several months ago on this list, I have added into the
>> > Disk Offerings Min, Max, and Burst values (all optional). These values
>> > follow the patterns of the Disk Size field in that they can be customized
>> > by the end user (in the Add Volume) if the admin decides to allow this.
>> >
>> > In the end, this feature allows users to create a 1:1 mapping between a
>> > volume on the SAN and a data disk that can be attached to a VM (XenServer
>> > supported...perhaps VMware, depending on time). This allows CloudStack
>> > admins to offer their end users guaranteed IOPS on data disks (eliminating
>> > the Noisy Neighbor effect). The plan is to support root disks in a post-4.2
>> > release.
>> >
>> > Again, I am sorry about my confusion regarding process here. I certainly
>> > appreciate all of the input I have received on the e-mail list over the
>> > past couple months while I was developing this feature.
>> >
>> > Please let me know if you have any questions.
>> >
>> > Thanks!
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud™
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud™
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
I think this is the link you'll need:
https://reviews.apache.org/r/11479/
Thanks! :)
On Fri, May 31, 2013 at 8:45 AM, John Burwell <jb...@basho.com> wrote:
> Mike/Animesh,
>
> I am happy to review. However, other $dayjob duties may prevent me from
> getting to it until Monday. Has the patch been pushed into Review Board?
>
> Thanks,
> -John
>
> On May 31, 2013, at 10:34 AM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> Hi Animesh,
>
> Sure, I'd be happy to do that.
>
> Would you like these comments in this e-mail chain, the design doc, and/or
> the JIRA ticket?
>
> Also, just to make sure I understand correctly when we refer to
> "dependencies" here, the plug-in simply requires the storage-framework
> changes that Edison put in place for 4.2. As long as those are in place
> (along with the enhancements I made to his storage framework), then the
> plug-in will function. There are no additional libraries used by the
> plug-in that were not already in use within CloudStack.
>
> Thanks!
>
>
> On Thu, May 30, 2013 at 10:31 PM, Animesh Chaturvedi <
> animesh.chaturvedi@citrix.com> wrote:
>
>> Mike don't feel bad it's my fault I think everyone knew you were working
>> on the plugin actively and I assumed the proposal was already in place.
>>
>> Can you call out your dependencies on Edison's framework?
>>
>> John I added as you as reviewers for Mike's contribution, can you pass on
>> your comments.
>>
>> Thanks
>> Animesh
>>
>> On May 30, 2013, at 9:09 PM, "Mike Tutkowski" <
>> mike.tutkowski@solidfire.com> wrote:
>>
>> > Hi everyone,
>> >
>> > I apologize for being unfamiliar with how I should have gone about
>> getting
>> > consensus for the storage plug-in I developed for 4.2.
>> >
>> > I talked with Animesh and he has asked me to send out this proposal
>> related
>> > to the storage plug-in I built for 4.2 and submitted to Review Board
>> > earlier this week.
>> >
>> > Here is a link to the design document:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
>> >
>> > Here is a link to the JIRA ticket:
>> >
>> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
>> >
>> > Here is a link to it on Review Board:
>> >
>> > https://reviews.apache.org/r/11479/
>> >
>> > Here is a high-level summary:
>> >
>> > I have developed a storage plug-in which makes use of the new storage
>> > framework that Edison put in place for the 4.2 release.
>> >
>> > Working with Edison, I have identified a few areas of the storage
>> framework
>> > that needed to be enhanced (and I have done so in the code that I
>> > submitted) for dynamic, zone-wide primary storage to function.
>> >
>> > This storage plug-in is specific to SolidFire (a data-storage company),
>> > whose architecture is designed around Quality of Service. Each volume on
>> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is,
>> as
>> > one might expect, a guaranteed number (even in fault conditions) (as
>> long
>> > as the IOPS of the SAN are not over-provisioned).
>> >
>> > Per a discussion several months ago on this list, I have added into the
>> > Disk Offerings Min, Max, and Burst values (all optional). These values
>> > follow the patterns of the Disk Size field in that they can be
>> customized
>> > by the end user (in the Add Volume) if the admin decides to allow this.
>> >
>> > In the end, this feature allows users to create a 1:1 mapping between a
>> > volume on the SAN and a data disk that can be attached to a VM
>> (XenServer
>> > supported...perhaps VMware, depending on time). This allows CloudStack
>> > admins to offer their end users guaranteed IOPS on data disks
>> (eliminating
>> > the Noisy Neighbor effect). The plan is to support root disks in a
>> post-4.2
>> > release.
>> >
>> > Again, I am sorry about my confusion regarding process here. I certainly
>> > appreciate all of the input I have received on the e-mail list over the
>> > past couple months while I was developing this feature.
>> >
>> > Please let me know if you have any questions.
>> >
>> > Thanks!
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by John Burwell <jb...@basho.com>.
Mike/Animesh,
I am happy to review. However, other $dayjob duties may prevent me from getting to it until Monday. Has the patch been pushed into Review Board?
Thanks,
-John
On May 31, 2013, at 10:34 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi Animesh,
>
> Sure, I'd be happy to do that.
>
> Would you like these comments in this e-mail chain, the design doc, and/or the JIRA ticket?
>
> Also, just to make sure I understand correctly when we refer to "dependencies" here, the plug-in simply requires the storage-framework changes that Edison put in place for 4.2. As long as those are in place (along with the enhancements I made to his storage framework), then the plug-in will function. There are no additional libraries used by the plug-in that were not already in use within CloudStack.
>
> Thanks!
>
>
> On Thu, May 30, 2013 at 10:31 PM, Animesh Chaturvedi <an...@citrix.com> wrote:
> Mike don't feel bad it's my fault I think everyone knew you were working on the plugin actively and I assumed the proposal was already in place.
>
> Can you call out your dependencies on Edison's framework?
>
> John I added as you as reviewers for Mike's contribution, can you pass on your comments.
>
> Thanks
> Animesh
>
> On May 30, 2013, at 9:09 PM, "Mike Tutkowski" <mi...@solidfire.com> wrote:
>
> > Hi everyone,
> >
> > I apologize for being unfamiliar with how I should have gone about getting
> > consensus for the storage plug-in I developed for 4.2.
> >
> > I talked with Animesh and he has asked me to send out this proposal related
> > to the storage plug-in I built for 4.2 and submitted to Review Board
> > earlier this week.
> >
> > Here is a link to the design document:
> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
> >
> > Here is a link to the JIRA ticket:
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
> >
> > Here is a link to it on Review Board:
> >
> > https://reviews.apache.org/r/11479/
> >
> > Here is a high-level summary:
> >
> > I have developed a storage plug-in which makes use of the new storage
> > framework that Edison put in place for the 4.2 release.
> >
> > Working with Edison, I have identified a few areas of the storage framework
> > that needed to be enhanced (and I have done so in the code that I
> > submitted) for dynamic, zone-wide primary storage to function.
> >
> > This storage plug-in is specific to SolidFire (a data-storage company),
> > whose architecture is designed around Quality of Service. Each volume on
> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
> > one might expect, a guaranteed number (even in fault conditions) (as long
> > as the IOPS of the SAN are not over-provisioned).
> >
> > Per a discussion several months ago on this list, I have added into the
> > Disk Offerings Min, Max, and Burst values (all optional). These values
> > follow the patterns of the Disk Size field in that they can be customized
> > by the end user (in the Add Volume) if the admin decides to allow this.
> >
> > In the end, this feature allows users to create a 1:1 mapping between a
> > volume on the SAN and a data disk that can be attached to a VM (XenServer
> > supported...perhaps VMware, depending on time). This allows CloudStack
> > admins to offer their end users guaranteed IOPS on data disks (eliminating
> > the Noisy Neighbor effect). The plan is to support root disks in a post-4.2
> > release.
> >
> > Again, I am sorry about my confusion regarding process here. I certainly
> > appreciate all of the input I have received on the e-mail list over the
> > past couple months while I was developing this feature.
> >
> > Please let me know if you have any questions.
> >
> > Thanks!
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud™
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Animesh,
Sure, I'd be happy to do that.
Would you like these comments in this e-mail chain, the design doc, and/or
the JIRA ticket?
Also, just to make sure I understand correctly when we refer to
"dependencies" here, the plug-in simply requires the storage-framework
changes that Edison put in place for 4.2. As long as those are in place
(along with the enhancements I made to his storage framework), then the
plug-in will function. There are no additional libraries used by the
plug-in that were not already in use within CloudStack.
Thanks!
On Thu, May 30, 2013 at 10:31 PM, Animesh Chaturvedi <
animesh.chaturvedi@citrix.com> wrote:
> Mike don't feel bad it's my fault I think everyone knew you were working
> on the plugin actively and I assumed the proposal was already in place.
>
> Can you call out your dependencies on Edison's framework?
>
> John I added as you as reviewers for Mike's contribution, can you pass on
> your comments.
>
> Thanks
> Animesh
>
> On May 30, 2013, at 9:09 PM, "Mike Tutkowski" <
> mike.tutkowski@solidfire.com> wrote:
>
> > Hi everyone,
> >
> > I apologize for being unfamiliar with how I should have gone about
> getting
> > consensus for the storage plug-in I developed for 4.2.
> >
> > I talked with Animesh and he has asked me to send out this proposal
> related
> > to the storage plug-in I built for 4.2 and submitted to Review Board
> > earlier this week.
> >
> > Here is a link to the design document:
> >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
> >
> > Here is a link to the JIRA ticket:
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-2778
> >
> > Here is a link to it on Review Board:
> >
> > https://reviews.apache.org/r/11479/
> >
> > Here is a high-level summary:
> >
> > I have developed a storage plug-in which makes use of the new storage
> > framework that Edison put in place for the 4.2 release.
> >
> > Working with Edison, I have identified a few areas of the storage
> framework
> > that needed to be enhanced (and I have done so in the code that I
> > submitted) for dynamic, zone-wide primary storage to function.
> >
> > This storage plug-in is specific to SolidFire (a data-storage company),
> > whose architecture is designed around Quality of Service. Each volume on
> > this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
> > one might expect, a guaranteed number (even in fault conditions) (as long
> > as the IOPS of the SAN are not over-provisioned).
> >
> > Per a discussion several months ago on this list, I have added into the
> > Disk Offerings Min, Max, and Burst values (all optional). These values
> > follow the patterns of the Disk Size field in that they can be customized
> > by the end user (in the Add Volume) if the admin decides to allow this.
> >
> > In the end, this feature allows users to create a 1:1 mapping between a
> > volume on the SAN and a data disk that can be attached to a VM (XenServer
> > supported...perhaps VMware, depending on time). This allows CloudStack
> > admins to offer their end users guaranteed IOPS on data disks
> (eliminating
> > the Noisy Neighbor effect). The plan is to support root disks in a
> post-4.2
> > release.
> >
> > Again, I am sorry about my confusion regarding process here. I certainly
> > appreciate all of the input I have received on the e-mail list over the
> > past couple months while I was developing this feature.
> >
> > Please let me know if you have any questions.
> >
> > Thanks!
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [PROPOSAL] Storage plug-in for 4.2
Posted by Animesh Chaturvedi <an...@citrix.com>.
Mike don't feel bad it's my fault I think everyone knew you were working on the plugin actively and I assumed the proposal was already in place.
Can you call out your dependencies on Edison's framework?
John I added as you as reviewers for Mike's contribution, can you pass on your comments.
Thanks
Animesh
On May 30, 2013, at 9:09 PM, "Mike Tutkowski" <mi...@solidfire.com> wrote:
> Hi everyone,
>
> I apologize for being unfamiliar with how I should have gone about getting
> consensus for the storage plug-in I developed for 4.2.
>
> I talked with Animesh and he has asked me to send out this proposal related
> to the storage plug-in I built for 4.2 and submitted to Review Board
> earlier this week.
>
> Here is a link to the design document:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Implement+SolidFire+(storage)+plug-in+and+expose+control+of+IOPS+to+admins+and+end+users
>
> Here is a link to the JIRA ticket:
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-2778
>
> Here is a link to it on Review Board:
>
> https://reviews.apache.org/r/11479/
>
> Here is a high-level summary:
>
> I have developed a storage plug-in which makes use of the new storage
> framework that Edison put in place for the 4.2 release.
>
> Working with Edison, I have identified a few areas of the storage framework
> that needed to be enhanced (and I have done so in the code that I
> submitted) for dynamic, zone-wide primary storage to function.
>
> This storage plug-in is specific to SolidFire (a data-storage company),
> whose architecture is designed around Quality of Service. Each volume on
> this SAN is assigned a Min, Max, and Burst number of IOPS. The Min is, as
> one might expect, a guaranteed number (even in fault conditions) (as long
> as the IOPS of the SAN are not over-provisioned).
>
> Per a discussion several months ago on this list, I have added into the
> Disk Offerings Min, Max, and Burst values (all optional). These values
> follow the patterns of the Disk Size field in that they can be customized
> by the end user (in the Add Volume) if the admin decides to allow this.
>
> In the end, this feature allows users to create a 1:1 mapping between a
> volume on the SAN and a data disk that can be attached to a VM (XenServer
> supported...perhaps VMware, depending on time). This allows CloudStack
> admins to offer their end users guaranteed IOPS on data disks (eliminating
> the Noisy Neighbor effect). The plan is to support root disks in a post-4.2
> release.
>
> Again, I am sorry about my confusion regarding process here. I certainly
> appreciate all of the input I have received on the e-mail list over the
> past couple months while I was developing this feature.
>
> Please let me know if you have any questions.
>
> Thanks!
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*