You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Wei ZHOU <us...@gmail.com> on 2013/06/10 19:54:45 UTC
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Guys,
I would like to merge disk_io_throttling branch into master.
Please review the code on https://reviews.apache.org/r/11782
If nobody object, I will merge into master in 72 hours.
-Wei
2013/5/30 Wei ZHOU <us...@gmail.com>
> Hi,
> I would like to merge disk_io_throttling branch into master.
> If nobody object, I will merge into master in 48 hours.
> The purpose is :
>
> Virtual machines are running on the same storage device (local storage or
> share strage). Because of the rate limitation of device (such as iops), if
> one VM has large disk operation, it may affect the disk performance of
> other VMs running on the same storage device.
> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>
> The feature includes:
>
> (1) set the maximum rate of VMs (in disk_offering, and global
> configuration)
> (2) change the maximum rate of VMs
> (3) limit the disk rate (total bps and iops)
> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> FS (I will update later) :
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> Merge check list :-
>
> * Did you check the branch's RAT execution success?
> Yes
>
> * Are there new dependencies introduced?
> No
>
> * What automated testing (unit and integration) is included in the new
> feature?
> Unit tests are added.
>
> * What testing has been done to check for potential regressions?
> (1) set the bytes rate and IOPS rate on CloudStack UI.
> (2) VM operations, including
> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> (3) Volume operations, including
> Attach, Detach
>
> To review the code, you can try
> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> f2e5591b710d04cc86815044f5823e73a4a58944
>
> Best regards,
> Wei
>
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> [2] refs/heads/disk_io_throttling
> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301 - VM Disk I/O Throttling)
>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
What if you specified multiple storage tags that mapped to different
storage vendors' storage?
I'm not sure how they could enter fields for all of those vendors if you
can only select one vendors' fields.
On Mon, Jun 10, 2013 at 2:21 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I'm not sure how that would work if the user picks SolidFire, but then
> specifies a Storage Tag that doesn't include SolidFire.
>
> As far as I know, Min, Max, and Burst is a superset for current storage
> vendors that do provisioned IOPS of some sort.
>
>
> On Mon, Jun 10, 2013 at 1:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>
>> Mike,
>> I do not think users can select only one of them, as they are implemented
>> on different sides.
>> Have you investigated the parameters other storage devices support,
>> besides
>> min/max/burst IOPS? You'd better add all possible fields in your
>> implementation.
>>
>> What do you think about this?
>> Hypersivor IOPS is fixed, and there is a drop-down box which includes all
>> supported storage vendors.
>> If users select "SolidFire", min/max/burst IOPS will appear.
>> If users select other vendors, relevant fields will appear.
>> Actually I still insist that it is better to add the storage-related
>> fields
>> in another table.
>>
>> -Wei
>>
>>
>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>>
>> > Here is my thinking:
>> >
>> > Two radio buttons (whatever we want to call them):
>> >
>> > 1) Hypervisor IOPS
>> > 2) Storage IOPS
>> >
>> > Leave them both un-checked by default.
>> >
>> > If the user checks one or the other, the relevant fields appear.
>> >
>> >
>> > On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com> wrote:
>> >
>> > > What do you think, Wei?
>> > >
>> > > Should we come up with a way for only one feature (yours or mine) to
>> be
>> > > used at a time on the new Disk Offering dialog?
>> > >
>> > > Since most storage-side provisioned IOPS don't break it down into
>> > separate
>> > > read and write categories, I think that's the way to go (only one
>> feature
>> > > or the other).
>> > >
>> > > Any suggestions from a usability standpoint how we want to implement
>> > this?
>> > > It could be as simple as a radio button to turn on your feature and
>> mine
>> > > off or vice versa.
>> > >
>> > > Thanks!
>> > >
>> > >
>> > > On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>> > wrote:
>> > >
>> > >> Mike,
>> > >>
>> > >> I agree -- I can't image a situation where you would want to use IOPS
>> > >> provisioned by both the hypervisor and storage. There are two
>> points of
>> > >> concern -- the UI and the management server. We have to ensure that
>> the
>> > >> user can't create a VM from a compute/disk offering combination where
>> > >> hypervisor throttled I/O would contradict/conflict with storage
>> > provisioned
>> > >> IOPS. I think this functional conflict must be resolved in the
>> > management
>> > >> server to ensure that API calls are properly validated with a UX that
>> > >> avoids user confusion. Have Wei and you worked out an approach to
>> > >> resolving this conflict?
>> > >>
>> > >> Thanks,
>> > >> -John
>> > >>
>> > >> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com>
>> > >> wrote:
>> > >>
>> > >> > Wei has sent me the screen shots.
>> > >> >
>> > >> > I don't support Compute Offerings for 4.2, so that's not an issue
>> > here.
>> > >> >
>> > >> > I do support Disk Offerings.
>> > >> >
>> > >> > It looks like Wei has added four new fields to the Disk Offering.
>> > >> >
>> > >> > I have added three (Min, Max, and Burst IOPS).
>> > >> >
>> > >> > We just need to decide if we should toggle between his and mine.
>> > >> >
>> > >> > I doubt a user would want to use both features at the same time.
>> > >> >
>> > >> >
>> > >> > On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jburwell@basho.com
>> >
>> > >> wrote:
>> > >> >
>> > >> >> Mike,
>> > >> >>
>> > >> >> Have Wei and you figured out the system level as well (e.g.
>> allowing
>> > >> >> either storage provisioned IOPS or hypervisor throttling, but no
>> > both)?
>> > >> >>
>> > >> >> Thanks,
>> > >> >> -John
>> > >> >>
>> > >> >> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>> > >> mike.tutkowski@solidfire.com>
>> > >> >> wrote:
>> > >> >>
>> > >> >>> Perhaps Wei could send me some screen shots of what he's changed
>> in
>> > >> the
>> > >> >> GUI
>> > >> >>> for his feature?
>> > >> >>>
>> > >> >>> Thanks!
>> > >> >>>
>> > >> >>>
>> > >> >>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>> jburwell@basho.com>
>> > >> >> wrote:
>> > >> >>>
>> > >> >>>> Wei,
>> > >> >>>>
>> > >> >>>> Have Mike Tutkowski and you reconciled the potential conflict
>> > >> between a
>> > >> >>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>> > solution
>> > >> >> did
>> > >> >>>> you select?
>> > >> >>>>
>> > >> >>>> Thanks,
>> > >> >>>> -John
>> > >> >>>>
>> > >> >>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>> > wrote:
>> > >> >>>>
>> > >> >>>>> Guys,
>> > >> >>>>>
>> > >> >>>>> I would like to merge disk_io_throttling branch into master.
>> > >> >>>>> Please review the code on https://reviews.apache.org/r/11782
>> > >> >>>>>
>> > >> >>>>> If nobody object, I will merge into master in 72 hours.
>> > >> >>>>>
>> > >> >>>>> -Wei
>> > >> >>>>>
>> > >> >>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>> > >> >>>>>
>> > >> >>>>>> Hi,
>> > >> >>>>>> I would like to merge disk_io_throttling branch into master.
>> > >> >>>>>> If nobody object, I will merge into master in 48 hours.
>> > >> >>>>>> The purpose is :
>> > >> >>>>>>
>> > >> >>>>>> Virtual machines are running on the same storage device (local
>> > >> storage
>> > >> >>>> or
>> > >> >>>>>> share strage). Because of the rate limitation of device (such
>> as
>> > >> >> iops),
>> > >> >>>> if
>> > >> >>>>>> one VM has large disk operation, it may affect the disk
>> > >> performance of
>> > >> >>>>>> other VMs running on the same storage device.
>> > >> >>>>>> It is neccesary to set the maximum rate and limit the disk
>> I/O of
>> > >> VMs.
>> > >> >>>>>>
>> > >> >>>>>> The feature includes:
>> > >> >>>>>>
>> > >> >>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>> > >> >>>>>> configuration)
>> > >> >>>>>> (2) change the maximum rate of VMs
>> > >> >>>>>> (3) limit the disk rate (total bps and iops)
>> > >> >>>>>> JIRA ticket:
>> > https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>> > >> >>>>>> FS (I will update later) :
>> > >> >>>>>>
>> > >> >>>>
>> > >> >>
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> > >> >>>>>> Merge check list :-
>> > >> >>>>>>
>> > >> >>>>>> * Did you check the branch's RAT execution success?
>> > >> >>>>>> Yes
>> > >> >>>>>>
>> > >> >>>>>> * Are there new dependencies introduced?
>> > >> >>>>>> No
>> > >> >>>>>>
>> > >> >>>>>> * What automated testing (unit and integration) is included in
>> > the
>> > >> new
>> > >> >>>>>> feature?
>> > >> >>>>>> Unit tests are added.
>> > >> >>>>>>
>> > >> >>>>>> * What testing has been done to check for potential
>> regressions?
>> > >> >>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> > >> >>>>>> (2) VM operations, including
>> > >> >>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>> restore
>> > >> >>>>>> (3) Volume operations, including
>> > >> >>>>>> Attach, Detach
>> > >> >>>>>>
>> > >> >>>>>> To review the code, you can try
>> > >> >>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>> > >> >>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>> > >> >>>>>>
>> > >> >>>>>> Best regards,
>> > >> >>>>>> Wei
>> > >> >>>>>>
>> > >> >>>>>> [1]
>> > >> >>>>>>
>> > >> >>>>
>> > >> >>
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> > >> >>>>>> [2] refs/heads/disk_io_throttling
>> > >> >>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>> > >> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>> > >> >(CLOUDSTACK-1301
>> > >> >> -
>> > >> >>>> VM Disk I/O Throttling)
>> > >> >>>>>>
>> > >> >>>>
>> > >> >>>>
>> > >> >>>
>> > >> >>>
>> > >> >>> --
>> > >> >>> *Mike Tutkowski*
>> > >> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >> >>> e: mike.tutkowski@solidfire.com
>> > >> >>> o: 303.746.7302
>> > >> >>> Advancing the way the world uses the
>> > >> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >> >>> *™*
>> > >> >>
>> > >> >>
>> > >> >
>> > >> >
>> > >> > --
>> > >> > *Mike Tutkowski*
>> > >> > *Senior CloudStack Developer, SolidFire Inc.*
>> > >> > e: mike.tutkowski@solidfire.com
>> > >> > o: 303.746.7302
>> > >> > Advancing the way the world uses the
>> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > >> > *™*
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > *Mike Tutkowski*
>> > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > e: mike.tutkowski@solidfire.com
>> > > o: 303.746.7302
>> > > Advancing the way the world uses the cloud<
>> > http://solidfire.com/solution/overview/?video=play>
>> > > *™*
>> > >
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>> >
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
I'm not sure how that would work if the user picks SolidFire, but then
specifies a Storage Tag that doesn't include SolidFire.
As far as I know, Min, Max, and Burst is a superset for current storage
vendors that do provisioned IOPS of some sort.
On Mon, Jun 10, 2013 at 1:59 PM, Wei ZHOU <us...@gmail.com> wrote:
> Mike,
> I do not think users can select only one of them, as they are implemented
> on different sides.
> Have you investigated the parameters other storage devices support, besides
> min/max/burst IOPS? You'd better add all possible fields in your
> implementation.
>
> What do you think about this?
> Hypersivor IOPS is fixed, and there is a drop-down box which includes all
> supported storage vendors.
> If users select "SolidFire", min/max/burst IOPS will appear.
> If users select other vendors, relevant fields will appear.
> Actually I still insist that it is better to add the storage-related fields
> in another table.
>
> -Wei
>
>
> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>
> > Here is my thinking:
> >
> > Two radio buttons (whatever we want to call them):
> >
> > 1) Hypervisor IOPS
> > 2) Storage IOPS
> >
> > Leave them both un-checked by default.
> >
> > If the user checks one or the other, the relevant fields appear.
> >
> >
> > On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> > > What do you think, Wei?
> > >
> > > Should we come up with a way for only one feature (yours or mine) to be
> > > used at a time on the new Disk Offering dialog?
> > >
> > > Since most storage-side provisioned IOPS don't break it down into
> > separate
> > > read and write categories, I think that's the way to go (only one
> feature
> > > or the other).
> > >
> > > Any suggestions from a usability standpoint how we want to implement
> > this?
> > > It could be as simple as a radio button to turn on your feature and
> mine
> > > off or vice versa.
> > >
> > > Thanks!
> > >
> > >
> > > On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >
> > >> Mike,
> > >>
> > >> I agree -- I can't image a situation where you would want to use IOPS
> > >> provisioned by both the hypervisor and storage. There are two points
> of
> > >> concern -- the UI and the management server. We have to ensure that
> the
> > >> user can't create a VM from a compute/disk offering combination where
> > >> hypervisor throttled I/O would contradict/conflict with storage
> > provisioned
> > >> IOPS. I think this functional conflict must be resolved in the
> > management
> > >> server to ensure that API calls are properly validated with a UX that
> > >> avoids user confusion. Have Wei and you worked out an approach to
> > >> resolving this conflict?
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >> > Wei has sent me the screen shots.
> > >> >
> > >> > I don't support Compute Offerings for 4.2, so that's not an issue
> > here.
> > >> >
> > >> > I do support Disk Offerings.
> > >> >
> > >> > It looks like Wei has added four new fields to the Disk Offering.
> > >> >
> > >> > I have added three (Min, Max, and Burst IOPS).
> > >> >
> > >> > We just need to decide if we should toggle between his and mine.
> > >> >
> > >> > I doubt a user would want to use both features at the same time.
> > >> >
> > >> >
> > >> > On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
> > >> wrote:
> > >> >
> > >> >> Mike,
> > >> >>
> > >> >> Have Wei and you figured out the system level as well (e.g.
> allowing
> > >> >> either storage provisioned IOPS or hypervisor throttling, but no
> > both)?
> > >> >>
> > >> >> Thanks,
> > >> >> -John
> > >> >>
> > >> >> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com>
> > >> >> wrote:
> > >> >>
> > >> >>> Perhaps Wei could send me some screen shots of what he's changed
> in
> > >> the
> > >> >> GUI
> > >> >>> for his feature?
> > >> >>>
> > >> >>> Thanks!
> > >> >>>
> > >> >>>
> > >> >>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
> jburwell@basho.com>
> > >> >> wrote:
> > >> >>>
> > >> >>>> Wei,
> > >> >>>>
> > >> >>>> Have Mike Tutkowski and you reconciled the potential conflict
> > >> between a
> > >> >>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> > solution
> > >> >> did
> > >> >>>> you select?
> > >> >>>>
> > >> >>>> Thanks,
> > >> >>>> -John
> > >> >>>>
> > >> >>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
> > wrote:
> > >> >>>>
> > >> >>>>> Guys,
> > >> >>>>>
> > >> >>>>> I would like to merge disk_io_throttling branch into master.
> > >> >>>>> Please review the code on https://reviews.apache.org/r/11782
> > >> >>>>>
> > >> >>>>> If nobody object, I will merge into master in 72 hours.
> > >> >>>>>
> > >> >>>>> -Wei
> > >> >>>>>
> > >> >>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> > >> >>>>>
> > >> >>>>>> Hi,
> > >> >>>>>> I would like to merge disk_io_throttling branch into master.
> > >> >>>>>> If nobody object, I will merge into master in 48 hours.
> > >> >>>>>> The purpose is :
> > >> >>>>>>
> > >> >>>>>> Virtual machines are running on the same storage device (local
> > >> storage
> > >> >>>> or
> > >> >>>>>> share strage). Because of the rate limitation of device (such
> as
> > >> >> iops),
> > >> >>>> if
> > >> >>>>>> one VM has large disk operation, it may affect the disk
> > >> performance of
> > >> >>>>>> other VMs running on the same storage device.
> > >> >>>>>> It is neccesary to set the maximum rate and limit the disk I/O
> of
> > >> VMs.
> > >> >>>>>>
> > >> >>>>>> The feature includes:
> > >> >>>>>>
> > >> >>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> > >> >>>>>> configuration)
> > >> >>>>>> (2) change the maximum rate of VMs
> > >> >>>>>> (3) limit the disk rate (total bps and iops)
> > >> >>>>>> JIRA ticket:
> > https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> > >> >>>>>> FS (I will update later) :
> > >> >>>>>>
> > >> >>>>
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >> >>>>>> Merge check list :-
> > >> >>>>>>
> > >> >>>>>> * Did you check the branch's RAT execution success?
> > >> >>>>>> Yes
> > >> >>>>>>
> > >> >>>>>> * Are there new dependencies introduced?
> > >> >>>>>> No
> > >> >>>>>>
> > >> >>>>>> * What automated testing (unit and integration) is included in
> > the
> > >> new
> > >> >>>>>> feature?
> > >> >>>>>> Unit tests are added.
> > >> >>>>>>
> > >> >>>>>> * What testing has been done to check for potential
> regressions?
> > >> >>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > >> >>>>>> (2) VM operations, including
> > >> >>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> > >> >>>>>> (3) Volume operations, including
> > >> >>>>>> Attach, Detach
> > >> >>>>>>
> > >> >>>>>> To review the code, you can try
> > >> >>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> > >> >>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> > >> >>>>>>
> > >> >>>>>> Best regards,
> > >> >>>>>> Wei
> > >> >>>>>>
> > >> >>>>>> [1]
> > >> >>>>>>
> > >> >>>>
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >> >>>>>> [2] refs/heads/disk_io_throttling
> > >> >>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> > >> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> > >> >(CLOUDSTACK-1301
> > >> >> -
> > >> >>>> VM Disk I/O Throttling)
> > >> >>>>>>
> > >> >>>>
> > >> >>>>
> > >> >>>
> > >> >>>
> > >> >>> --
> > >> >>> *Mike Tutkowski*
> > >> >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> >>> e: mike.tutkowski@solidfire.com
> > >> >>> o: 303.746.7302
> > >> >>> Advancing the way the world uses the
> > >> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> >>> *™*
> > >> >>
> > >> >>
> > >> >
> > >> >
> > >> > --
> > >> > *Mike Tutkowski*
> > >> > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > e: mike.tutkowski@solidfire.com
> > >> > o: 303.746.7302
> > >> > Advancing the way the world uses the
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > *™*
> > >>
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the cloud<
> > http://solidfire.com/solution/overview/?video=play>
> > > *™*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Mike,
I am asking whether or not we can defer the entire notion of Burst IOPS in the data model (e.g. the driver could default it to Max IOPS when configuring a SAN volume) for 4.2. In 4.3, we design a facility for extended driver data that would allow drivers to expose these implementation-specific parameters in a generic manner. Using this facility, we would enhance the SolidFire driver to support Burst IOPS configuration.
Thanks,
-John
On Jun 10, 2013, at 5:52 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Just to make sure I understand your request, are you looking to display Min
> and Max (as long as Wei's feature is not in use), but not display Burst
> IOPS?
>
>
> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> My concern becomes that we start ballooning the data model and user
>> interface with a fields that are documented as, "If using SolidFire, then
>> Burst IOPS is honored and foo and bar are not. For solution X, Burst IOPS
>> is ignored, but foo and bar apply." It may have to hold until 4.3, but it
>> seems like we need an extended data concept for storage drivers that allow
>> them to define an additional set of properties, and persist them into the
>> database as a JSON document. Such an enhancement would also require some
>> UI fanciness to consume the metadata provided by the driver and adjust the
>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
>> address extended driver data in a holistic manner?
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> My thinking is that Min and Max are industry standard and Burst is a new
>>> concept that could catch on.
>>>
>>>
>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Wei,
>>>>
>>>> In this case, we can have the hypervisor or storage providing the
>> quality
>>>> of service guarantee. Naively, it seems reasonable to separate
>> hypervisor
>>>> and storage QoS parameters and enforce mutually exclusion. Not only
>> does
>>>> this approach simplify a whole range of operations, it also avoids
>>>> reconciling the data models. The only question I have about the storage
>>>> provision IOPS is whether or not "Burst IOPS" is an industry standard or
>>>> vendor specific concept.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>>
>>>>> Mike,
>>>>> I do not think users can select only one of them, as they are
>> implemented
>>>>> on different sides.
>>>>> Have you investigated the parameters other storage devices support,
>>>> besides
>>>>> min/max/burst IOPS? You'd better add all possible fields in your
>>>>> implementation.
>>>>>
>>>>> What do you think about this?
>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which includes
>> all
>>>>> supported storage vendors.
>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
>>>>> If users select other vendors, relevant fields will appear.
>>>>> Actually I still insist that it is better to add the storage-related
>>>> fields
>>>>> in another table.
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>>>>>
>>>>>> Here is my thinking:
>>>>>>
>>>>>> Two radio buttons (whatever we want to call them):
>>>>>>
>>>>>> 1) Hypervisor IOPS
>>>>>> 2) Storage IOPS
>>>>>>
>>>>>> Leave them both un-checked by default.
>>>>>>
>>>>>> If the user checks one or the other, the relevant fields appear.
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> What do you think, Wei?
>>>>>>>
>>>>>>> Should we come up with a way for only one feature (yours or mine) to
>> be
>>>>>>> used at a time on the new Disk Offering dialog?
>>>>>>>
>>>>>>> Since most storage-side provisioned IOPS don't break it down into
>>>>>> separate
>>>>>>> read and write categories, I think that's the way to go (only one
>>>> feature
>>>>>>> or the other).
>>>>>>>
>>>>>>> Any suggestions from a usability standpoint how we want to implement
>>>>>> this?
>>>>>>> It could be as simple as a radio button to turn on your feature and
>>>> mine
>>>>>>> off or vice versa.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> I agree -- I can't image a situation where you would want to use
>> IOPS
>>>>>>>> provisioned by both the hypervisor and storage. There are two
>> points
>>>> of
>>>>>>>> concern -- the UI and the management server. We have to ensure that
>>>> the
>>>>>>>> user can't create a VM from a compute/disk offering combination
>> where
>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage
>>>>>> provisioned
>>>>>>>> IOPS. I think this functional conflict must be resolved in the
>>>>>> management
>>>>>>>> server to ensure that API calls are properly validated with a UX
>> that
>>>>>>>> avoids user confusion. Have Wei and you worked out an approach to
>>>>>>>> resolving this conflict?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Wei has sent me the screen shots.
>>>>>>>>>
>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an issue
>>>>>> here.
>>>>>>>>>
>>>>>>>>> I do support Disk Offerings.
>>>>>>>>>
>>>>>>>>> It looks like Wei has added four new fields to the Disk Offering.
>>>>>>>>>
>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>>>>>
>>>>>>>>> We just need to decide if we should toggle between his and mine.
>>>>>>>>>
>>>>>>>>> I doubt a user would want to use both features at the same time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jburwell@basho.com
>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> Have Wei and you figured out the system level as well (e.g.
>> allowing
>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but no
>>>>>> both)?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's changed
>> in
>>>>>>>> the
>>>>>>>>>> GUI
>>>>>>>>>>> for his feature?
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>> jburwell@basho.com
>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Wei,
>>>>>>>>>>>>
>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>>>>>>>> between a
>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>>>>>> solution
>>>>>>>>>> did
>>>>>>>>>>>> you select?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
>>>>>>>>>>>>>
>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Virtual machines are running on the same storage device (local
>>>>>>>> storage
>>>>>>>>>>>> or
>>>>>>>>>>>>>> share strage). Because of the rate limitation of device (such
>> as
>>>>>>>>>> iops),
>>>>>>>>>>>> if
>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>>> performance of
>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
>>>> of
>>>>>>>> VMs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>> JIRA ticket:
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * What automated testing (unit and integration) is included in
>>>>>> the
>>>>>>>> new
>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * What testing has been done to check for potential
>> regressions?
>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>> restore
>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>>>>>> (CLOUDSTACK-1301
>>>>>>>>>> -
>>>>>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *™*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *™*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *™*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *™*
>>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, I think it's fine for Wei to merge in his changes. I can then fetch
and merge into my branch and add the additional GUI and API code for mutual
exclusion.
On Thu, Jun 13, 2013 at 9:34 AM, Wei ZHOU <us...@gmail.com> wrote:
> John,
>
> Please review the code on https://reviews.apache.org/r/11782
> The storage provisioned IOPS does not affect hypervisor throttled I/O, I
> think.
> Mike may change UI and java code for storage provisioned IOPS after the
> merge.
>
> -Wei
>
>
> 2013/6/13 John Burwell <jb...@basho.com>
>
> > Wei,
> >
> > There are open questions on the thread regarding mutual exclusion of
> > hypervisor throttled I/O and storage provisioned IOPS. We need to
> > understand how and where it will be implemented in both the UI and
> > service layers. Also, can you resend the Review Board review? My
> > email search skills have failed to find it.
> >
> > Thanks,
> > -John
> >
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, I did that. :)
I had to change some allocator code, too, because it didn't like zone-wide
storage being set to hypervisor any.
On Thu, Jun 13, 2013 at 3:45 PM, Edison Su <Ed...@citrix.com> wrote:
> How about set hypervisorType to Any? Haven’t take a look at the master
> change yet.****
>
> ** **
>
> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> *Sent:* Thursday, June 13, 2013 1:41 PM
> *To:* dev@cloudstack.apache.org
> *Cc:* Edison Su
> *Subject:* Re: [MERGE] disk_io_throttling to MASTER (Second Round)****
>
> ** **
>
> Actually, I am noticing some new behavior around picking a storage pool
> for zone-wide storage.****
>
> ** **
>
> The current implementation that I've brought down from master no longer
> finds storage for me because my plug-in is zone wide and not associated
> with a hypervisor.****
>
> ** **
>
> Edison?****
>
> ** **
>
> On Thu, Jun 13, 2013 at 1:13 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:****
>
> Hi Edison,****
>
> ** **
>
> I notice after I updated from master that Hypervisor Type is now a
> required parameter for creating a storage pool.****
>
> ** **
>
> Should I just use HypervisorType.Any?****
>
> ** **
>
> Thanks!****
>
> ** **
>
> On Thu, Jun 13, 2013 at 12:21 PM, John Burwell <jb...@basho.com> wrote:
> ****
>
> Wei,
>
> I published my review. I didn't see any code to validating the rate
> values (e.g. values greater than 0, values less than an maximum value).
> Did I miss it?
>
> I also noticed that 0 is being used when no value has been specified. I
> recommend using the Long type rather primitive long in order to use null to
> represent unspecified values rather than a magic value.
>
> Thanks,
> -John****
>
>
> On Jun 13, 2013, at 11:34 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> > John,
> >
> > Please review the code on https://reviews.apache.org/r/11782
> > The storage provisioned IOPS does not affect hypervisor throttled I/O, I
> > think.
> > Mike may change UI and java code for storage provisioned IOPS after the
> > merge.
> >
> > -Wei
> >
> >
> > 2013/6/13 John Burwell <jb...@basho.com>
> >
> >> Wei,
> >>
> >> There are open questions on the thread regarding mutual exclusion of
> >> hypervisor throttled I/O and storage provisioned IOPS. We need to
> >> understand how and where it will be implemented in both the UI and
> >> service layers. Also, can you resend the Review Board review? My
> >> email search skills have failed to find it.
> >>
> >> Thanks,
> >> -John
> >>
> >>****
>
>
>
> ****
>
> ** **
>
> --
> *Mike Tutkowski*****
>
> *Senior CloudStack Developer, SolidFire Inc.*****
>
> e: mike.tutkowski@solidfire.com****
>
> o: 303.746.7302****
>
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*****
>
>
>
> ****
>
> ** **
>
> --
> *Mike Tutkowski*****
>
> *Senior CloudStack Developer, SolidFire Inc.*****
>
> e: mike.tutkowski@solidfire.com****
>
> o: 303.746.7302****
>
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*****
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
RE: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Edison Su <Ed...@citrix.com>.
How about set hypervisorType to Any? Haven't take a look at the master change yet.
From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
Sent: Thursday, June 13, 2013 1:41 PM
To: dev@cloudstack.apache.org
Cc: Edison Su
Subject: Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Actually, I am noticing some new behavior around picking a storage pool for zone-wide storage.
The current implementation that I've brought down from master no longer finds storage for me because my plug-in is zone wide and not associated with a hypervisor.
Edison?
On Thu, Jun 13, 2013 at 1:13 PM, Mike Tutkowski <mi...@solidfire.com>> wrote:
Hi Edison,
I notice after I updated from master that Hypervisor Type is now a required parameter for creating a storage pool.
Should I just use HypervisorType.Any?
Thanks!
On Thu, Jun 13, 2013 at 12:21 PM, John Burwell <jb...@basho.com>> wrote:
Wei,
I published my review. I didn't see any code to validating the rate values (e.g. values greater than 0, values less than an maximum value). Did I miss it?
I also noticed that 0 is being used when no value has been specified. I recommend using the Long type rather primitive long in order to use null to represent unspecified values rather than a magic value.
Thanks,
-John
On Jun 13, 2013, at 11:34 AM, Wei ZHOU <us...@gmail.com>> wrote:
> John,
>
> Please review the code on https://reviews.apache.org/r/11782
> The storage provisioned IOPS does not affect hypervisor throttled I/O, I
> think.
> Mike may change UI and java code for storage provisioned IOPS after the
> merge.
>
> -Wei
>
>
> 2013/6/13 John Burwell <jb...@basho.com>>
>
>> Wei,
>>
>> There are open questions on the thread regarding mutual exclusion of
>> hypervisor throttled I/O and storage provisioned IOPS. We need to
>> understand how and where it will be implemented in both the UI and
>> service layers. Also, can you resend the Review Board review? My
>> email search skills have failed to find it.
>>
>> Thanks,
>> -John
>>
>>
--
Mike Tutkowski
Senior CloudStack Developer, SolidFire Inc.
e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
o: 303.746.7302<tel:303.746.7302>
Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm)
--
Mike Tutkowski
Senior CloudStack Developer, SolidFire Inc.
e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
o: 303.746.7302
Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm)
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Actually, I am noticing some new behavior around picking a storage pool for
zone-wide storage.
The current implementation that I've brought down from master no longer
finds storage for me because my plug-in is zone wide and not associated
with a hypervisor.
Edison?
On Thu, Jun 13, 2013 at 1:13 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Hi Edison,
>
> I notice after I updated from master that Hypervisor Type is now a
> required parameter for creating a storage pool.
>
> Should I just use HypervisorType.Any?
>
> Thanks!
>
>
> On Thu, Jun 13, 2013 at 12:21 PM, John Burwell <jb...@basho.com> wrote:
>
>> Wei,
>>
>> I published my review. I didn't see any code to validating the rate
>> values (e.g. values greater than 0, values less than an maximum value).
>> Did I miss it?
>>
>> I also noticed that 0 is being used when no value has been specified. I
>> recommend using the Long type rather primitive long in order to use null to
>> represent unspecified values rather than a magic value.
>>
>> Thanks,
>> -John
>>
>> On Jun 13, 2013, at 11:34 AM, Wei ZHOU <us...@gmail.com> wrote:
>>
>> > John,
>> >
>> > Please review the code on https://reviews.apache.org/r/11782
>> > The storage provisioned IOPS does not affect hypervisor throttled I/O, I
>> > think.
>> > Mike may change UI and java code for storage provisioned IOPS after the
>> > merge.
>> >
>> > -Wei
>> >
>> >
>> > 2013/6/13 John Burwell <jb...@basho.com>
>> >
>> >> Wei,
>> >>
>> >> There are open questions on the thread regarding mutual exclusion of
>> >> hypervisor throttled I/O and storage provisioned IOPS. We need to
>> >> understand how and where it will be implemented in both the UI and
>> >> service layers. Also, can you resend the Review Board review? My
>> >> email search skills have failed to find it.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Edison,
I notice after I updated from master that Hypervisor Type is now a required
parameter for creating a storage pool.
Should I just use HypervisorType.Any?
Thanks!
On Thu, Jun 13, 2013 at 12:21 PM, John Burwell <jb...@basho.com> wrote:
> Wei,
>
> I published my review. I didn't see any code to validating the rate
> values (e.g. values greater than 0, values less than an maximum value).
> Did I miss it?
>
> I also noticed that 0 is being used when no value has been specified. I
> recommend using the Long type rather primitive long in order to use null to
> represent unspecified values rather than a magic value.
>
> Thanks,
> -John
>
> On Jun 13, 2013, at 11:34 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> > John,
> >
> > Please review the code on https://reviews.apache.org/r/11782
> > The storage provisioned IOPS does not affect hypervisor throttled I/O, I
> > think.
> > Mike may change UI and java code for storage provisioned IOPS after the
> > merge.
> >
> > -Wei
> >
> >
> > 2013/6/13 John Burwell <jb...@basho.com>
> >
> >> Wei,
> >>
> >> There are open questions on the thread regarding mutual exclusion of
> >> hypervisor throttled I/O and storage provisioned IOPS. We need to
> >> understand how and where it will be implemented in both the UI and
> >> service layers. Also, can you resend the Review Board review? My
> >> email search skills have failed to find it.
> >>
> >> Thanks,
> >> -John
> >>
> >>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Wei,
I published my review. I didn't see any code to validating the rate values (e.g. values greater than 0, values less than an maximum value). Did I miss it?
I also noticed that 0 is being used when no value has been specified. I recommend using the Long type rather primitive long in order to use null to represent unspecified values rather than a magic value.
Thanks,
-John
On Jun 13, 2013, at 11:34 AM, Wei ZHOU <us...@gmail.com> wrote:
> John,
>
> Please review the code on https://reviews.apache.org/r/11782
> The storage provisioned IOPS does not affect hypervisor throttled I/O, I
> think.
> Mike may change UI and java code for storage provisioned IOPS after the
> merge.
>
> -Wei
>
>
> 2013/6/13 John Burwell <jb...@basho.com>
>
>> Wei,
>>
>> There are open questions on the thread regarding mutual exclusion of
>> hypervisor throttled I/O and storage provisioned IOPS. We need to
>> understand how and where it will be implemented in both the UI and
>> service layers. Also, can you resend the Review Board review? My
>> email search skills have failed to find it.
>>
>> Thanks,
>> -John
>>
>>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Wei ZHOU <us...@gmail.com>.
John,
Please review the code on https://reviews.apache.org/r/11782
The storage provisioned IOPS does not affect hypervisor throttled I/O, I
think.
Mike may change UI and java code for storage provisioned IOPS after the
merge.
-Wei
2013/6/13 John Burwell <jb...@basho.com>
> Wei,
>
> There are open questions on the thread regarding mutual exclusion of
> hypervisor throttled I/O and storage provisioned IOPS. We need to
> understand how and where it will be implemented in both the UI and
> service layers. Also, can you resend the Review Board review? My
> email search skills have failed to find it.
>
> Thanks,
> -John
>
>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Wei,
There are open questions on the thread regarding mutual exclusion of
hypervisor throttled I/O and storage provisioned IOPS. We need to
understand how and where it will be implemented in both the UI and
service layers. Also, can you resend the Review Board review? My
email search skills have failed to find it.
Thanks,
-John
On Jun 13, 2013, at 3:15 AM, Wei ZHOU <us...@gmail.com> wrote:
> If nobody object, I will merge into master today.
>
> -Wei
>
>
> 2013/6/11 John Burwell <jb...@basho.com>
>
>> Mike,
>>
>> From a CloudStack perspective, it will keep implementation specific
>> concepts from the base data model, and provide a great test case to develop
>> a mechanism to capture this information in 4.3. Ideally, I want CloudStack
>> to exploit these implementation specific features. I just want to provide
>> a facility to manage that data that does not leak across implementations.
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 6:07 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> I am pretty sure it is not a blocker for me to drop Burst IOPS. Let me
>> run
>>> that past Product Management, but I don't think it's a problem.
>>>
>>>
>>> On Mon, Jun 10, 2013 at 4:01 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Yes. I realize my other reply did not explicitly state leaving the Min
>>>> and Max IOPS fields in the data model as these seem to generic terms
>> across
>>>> storage implementations.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> More generally speaking, you're looking to remove Burst IOPS from
>>>>> CloudStack for 4.2, but we would keep Min and Max (and they would be
>>>>> displayed in the Disk Offering dialog as I've proposed)?
>>>>>
>>>>>
>>>>> On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Just to make sure I understand your request, are you looking to
>> display
>>>>>> Min and Max (as long as Wei's feature is not in use), but not display
>>>> Burst
>>>>>> IOPS?
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> My concern becomes that we start ballooning the data model and user
>>>>>>> interface with a fields that are documented as, "If using SolidFire,
>>>> then
>>>>>>> Burst IOPS is honored and foo and bar are not. For solution X, Burst
>>>> IOPS
>>>>>>> is ignored, but foo and bar apply." It may have to hold until 4.3,
>>>> but it
>>>>>>> seems like we need an extended data concept for storage drivers that
>>>> allow
>>>>>>> them to define an additional set of properties, and persist them into
>>>> the
>>>>>>> database as a JSON document. Such an enhancement would also require
>>>> some
>>>>>>> UI fanciness to consume the metadata provided by the driver and
>> adjust
>>>> the
>>>>>>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
>>>>>>> address extended driver data in a holistic manner?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> My thinking is that Min and Max are industry standard and Burst is a
>>>> new
>>>>>>>> concept that could catch on.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Wei,
>>>>>>>>>
>>>>>>>>> In this case, we can have the hypervisor or storage providing the
>>>>>>> quality
>>>>>>>>> of service guarantee. Naively, it seems reasonable to separate
>>>>>>> hypervisor
>>>>>>>>> and storage QoS parameters and enforce mutually exclusion. Not
>> only
>>>>>>> does
>>>>>>>>> this approach simplify a whole range of operations, it also avoids
>>>>>>>>> reconciling the data models. The only question I have about the
>>>>>>> storage
>>>>>>>>> provision IOPS is whether or not "Burst IOPS" is an industry
>> standard
>>>>>>> or
>>>>>>>>> vendor specific concept.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com>
>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>> I do not think users can select only one of them, as they are
>>>>>>> implemented
>>>>>>>>>> on different sides.
>>>>>>>>>> Have you investigated the parameters other storage devices
>> support,
>>>>>>>>> besides
>>>>>>>>>> min/max/burst IOPS? You'd better add all possible fields in your
>>>>>>>>>> implementation.
>>>>>>>>>>
>>>>>>>>>> What do you think about this?
>>>>>>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which
>>>> includes
>>>>>>> all
>>>>>>>>>> supported storage vendors.
>>>>>>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
>>>>>>>>>> If users select other vendors, relevant fields will appear.
>>>>>>>>>> Actually I still insist that it is better to add the
>> storage-related
>>>>>>>>> fields
>>>>>>>>>> in another table.
>>>>>>>>>>
>>>>>>>>>> -Wei
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>
>>>>>>>>>>> Here is my thinking:
>>>>>>>>>>>
>>>>>>>>>>> Two radio buttons (whatever we want to call them):
>>>>>>>>>>>
>>>>>>>>>>> 1) Hypervisor IOPS
>>>>>>>>>>> 2) Storage IOPS
>>>>>>>>>>>
>>>>>>>>>>> Leave them both un-checked by default.
>>>>>>>>>>>
>>>>>>>>>>> If the user checks one or the other, the relevant fields appear.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> What do you think, Wei?
>>>>>>>>>>>>
>>>>>>>>>>>> Should we come up with a way for only one feature (yours or
>> mine)
>>>>>>> to be
>>>>>>>>>>>> used at a time on the new Disk Offering dialog?
>>>>>>>>>>>>
>>>>>>>>>>>> Since most storage-side provisioned IOPS don't break it down
>> into
>>>>>>>>>>> separate
>>>>>>>>>>>> read and write categories, I think that's the way to go (only
>> one
>>>>>>>>> feature
>>>>>>>>>>>> or the other).
>>>>>>>>>>>>
>>>>>>>>>>>> Any suggestions from a usability standpoint how we want to
>>>> implement
>>>>>>>>>>> this?
>>>>>>>>>>>> It could be as simple as a radio button to turn on your feature
>>>> and
>>>>>>>>> mine
>>>>>>>>>>>> off or vice versa.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <
>> jburwell@basho.com
>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree -- I can't image a situation where you would want to
>> use
>>>>>>> IOPS
>>>>>>>>>>>>> provisioned by both the hypervisor and storage. There are two
>>>>>>> points
>>>>>>>>> of
>>>>>>>>>>>>> concern -- the UI and the management server. We have to ensure
>>>>>>> that
>>>>>>>>> the
>>>>>>>>>>>>> user can't create a VM from a compute/disk offering combination
>>>>>>> where
>>>>>>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage
>>>>>>>>>>> provisioned
>>>>>>>>>>>>> IOPS. I think this functional conflict must be resolved in the
>>>>>>>>>>> management
>>>>>>>>>>>>> server to ensure that API calls are properly validated with a
>> UX
>>>>>>> that
>>>>>>>>>>>>> avoids user confusion. Have Wei and you worked out an approach
>>>> to
>>>>>>>>>>>>> resolving this conflict?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wei has sent me the screen shots.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an
>>>> issue
>>>>>>>>>>> here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I do support Disk Offerings.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It looks like Wei has added four new fields to the Disk
>>>> Offering.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We just need to decide if we should toggle between his and
>> mine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I doubt a user would want to use both features at the same
>> time.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Have Wei and you figured out the system level as well (e.g.
>>>>>>> allowing
>>>>>>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but
>>>> no
>>>>>>>>>>> both)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's
>>>>>>> changed in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>> for his feature?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>>>>>>> jburwell@basho.com
>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential
>> conflict
>>>>>>>>>>>>> between a
>>>>>>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so,
>> what
>>>>>>>>>>> solution
>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>> you select?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <
>> ustcweizhou@gmail.com
>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>> master.
>>>>>>>>>>>>>>>>>> Please review the code on
>>>> https://reviews.apache.org/r/11782
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>>>> master.
>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>>>>>>> (local
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>>>>>>> (such as
>>>>>>>>>>>>>>> iops),
>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>>>>>>>> performance of
>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
>> disk
>>>>>>> I/O
>>>>>>>>> of
>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>>>> global
>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>> included
>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>>>>>> restore
>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1]
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>> [3]
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>>>>>>>>>>> (CLOUDSTACK-1301
>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *™*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *™*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *™*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *™*
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *™*
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *™*
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *™*
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *™*
>>
>>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Wei ZHOU <us...@gmail.com>.
If nobody object, I will merge into master today.
-Wei
2013/6/11 John Burwell <jb...@basho.com>
> Mike,
>
> From a CloudStack perspective, it will keep implementation specific
> concepts from the base data model, and provide a great test case to develop
> a mechanism to capture this information in 4.3. Ideally, I want CloudStack
> to exploit these implementation specific features. I just want to provide
> a facility to manage that data that does not leak across implementations.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 6:07 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > I am pretty sure it is not a blocker for me to drop Burst IOPS. Let me
> run
> > that past Product Management, but I don't think it's a problem.
> >
> >
> > On Mon, Jun 10, 2013 at 4:01 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Yes. I realize my other reply did not explicitly state leaving the Min
> >> and Max IOPS fields in the data model as these seem to generic terms
> across
> >> storage implementations.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> More generally speaking, you're looking to remove Burst IOPS from
> >>> CloudStack for 4.2, but we would keep Min and Max (and they would be
> >>> displayed in the Disk Offering dialog as I've proposed)?
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Just to make sure I understand your request, are you looking to
> display
> >>>> Min and Max (as long as Wei's feature is not in use), but not display
> >> Burst
> >>>> IOPS?
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> My concern becomes that we start ballooning the data model and user
> >>>>> interface with a fields that are documented as, "If using SolidFire,
> >> then
> >>>>> Burst IOPS is honored and foo and bar are not. For solution X, Burst
> >> IOPS
> >>>>> is ignored, but foo and bar apply." It may have to hold until 4.3,
> >> but it
> >>>>> seems like we need an extended data concept for storage drivers that
> >> allow
> >>>>> them to define an additional set of properties, and persist them into
> >> the
> >>>>> database as a JSON document. Such an enhancement would also require
> >> some
> >>>>> UI fanciness to consume the metadata provided by the driver and
> adjust
> >> the
> >>>>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
> >>>>> address extended driver data in a holistic manner?
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>>> My thinking is that Min and Max are industry standard and Burst is a
> >> new
> >>>>>> concept that could catch on.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Wei,
> >>>>>>>
> >>>>>>> In this case, we can have the hypervisor or storage providing the
> >>>>> quality
> >>>>>>> of service guarantee. Naively, it seems reasonable to separate
> >>>>> hypervisor
> >>>>>>> and storage QoS parameters and enforce mutually exclusion. Not
> only
> >>>>> does
> >>>>>>> this approach simplify a whole range of operations, it also avoids
> >>>>>>> reconciling the data models. The only question I have about the
> >>>>> storage
> >>>>>>> provision IOPS is whether or not "Burst IOPS" is an industry
> standard
> >>>>> or
> >>>>>>> vendor specific concept.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com>
> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>> I do not think users can select only one of them, as they are
> >>>>> implemented
> >>>>>>>> on different sides.
> >>>>>>>> Have you investigated the parameters other storage devices
> support,
> >>>>>>> besides
> >>>>>>>> min/max/burst IOPS? You'd better add all possible fields in your
> >>>>>>>> implementation.
> >>>>>>>>
> >>>>>>>> What do you think about this?
> >>>>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which
> >> includes
> >>>>> all
> >>>>>>>> supported storage vendors.
> >>>>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
> >>>>>>>> If users select other vendors, relevant fields will appear.
> >>>>>>>> Actually I still insist that it is better to add the
> storage-related
> >>>>>>> fields
> >>>>>>>> in another table.
> >>>>>>>>
> >>>>>>>> -Wei
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>
> >>>>>>>>> Here is my thinking:
> >>>>>>>>>
> >>>>>>>>> Two radio buttons (whatever we want to call them):
> >>>>>>>>>
> >>>>>>>>> 1) Hypervisor IOPS
> >>>>>>>>> 2) Storage IOPS
> >>>>>>>>>
> >>>>>>>>> Leave them both un-checked by default.
> >>>>>>>>>
> >>>>>>>>> If the user checks one or the other, the relevant fields appear.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> What do you think, Wei?
> >>>>>>>>>>
> >>>>>>>>>> Should we come up with a way for only one feature (yours or
> mine)
> >>>>> to be
> >>>>>>>>>> used at a time on the new Disk Offering dialog?
> >>>>>>>>>>
> >>>>>>>>>> Since most storage-side provisioned IOPS don't break it down
> into
> >>>>>>>>> separate
> >>>>>>>>>> read and write categories, I think that's the way to go (only
> one
> >>>>>>> feature
> >>>>>>>>>> or the other).
> >>>>>>>>>>
> >>>>>>>>>> Any suggestions from a usability standpoint how we want to
> >> implement
> >>>>>>>>> this?
> >>>>>>>>>> It could be as simple as a radio button to turn on your feature
> >> and
> >>>>>>> mine
> >>>>>>>>>> off or vice versa.
> >>>>>>>>>>
> >>>>>>>>>> Thanks!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> I agree -- I can't image a situation where you would want to
> use
> >>>>> IOPS
> >>>>>>>>>>> provisioned by both the hypervisor and storage. There are two
> >>>>> points
> >>>>>>> of
> >>>>>>>>>>> concern -- the UI and the management server. We have to ensure
> >>>>> that
> >>>>>>> the
> >>>>>>>>>>> user can't create a VM from a compute/disk offering combination
> >>>>> where
> >>>>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage
> >>>>>>>>> provisioned
> >>>>>>>>>>> IOPS. I think this functional conflict must be resolved in the
> >>>>>>>>> management
> >>>>>>>>>>> server to ensure that API calls are properly validated with a
> UX
> >>>>> that
> >>>>>>>>>>> avoids user confusion. Have Wei and you worked out an approach
> >> to
> >>>>>>>>>>> resolving this conflict?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Wei has sent me the screen shots.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an
> >> issue
> >>>>>>>>> here.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I do support Disk Offerings.
> >>>>>>>>>>>>
> >>>>>>>>>>>> It looks like Wei has added four new fields to the Disk
> >> Offering.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
> >>>>>>>>>>>>
> >>>>>>>>>>>> We just need to decide if we should toggle between his and
> mine.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I doubt a user would want to use both features at the same
> time.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Have Wei and you figured out the system level as well (e.g.
> >>>>> allowing
> >>>>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but
> >> no
> >>>>>>>>> both)?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's
> >>>>> changed in
> >>>>>>>>>>> the
> >>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>> for his feature?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
> >>>>> jburwell@basho.com
> >>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential
> conflict
> >>>>>>>>>>> between a
> >>>>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so,
> what
> >>>>>>>>> solution
> >>>>>>>>>>>>> did
> >>>>>>>>>>>>>>> you select?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <
> ustcweizhou@gmail.com
> >>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
> master.
> >>>>>>>>>>>>>>>> Please review the code on
> >> https://reviews.apache.org/r/11782
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
> >> master.
> >>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
> >>>>> (local
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
> >>>>> (such as
> >>>>>>>>>>>>> iops),
> >>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>>>>>>>>> performance of
> >>>>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
> disk
> >>>>> I/O
> >>>>>>> of
> >>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
> >> global
> >>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>> JIRA ticket:
> >>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
> included
> >>>>> in
> >>>>>>>>> the
> >>>>>>>>>>> new
> >>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * What testing has been done to check for potential
> >>>>> regressions?
> >>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> >>>>> restore
> >>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>> [3]
> https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> >>>>>>>>>>>> (CLOUDSTACK-1301
> >>>>>>>>>>>>> -
> >>>>>>>>>>>>>>> VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *™*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *™*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *™*
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *™*
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *™*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *™*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Mike,
From a CloudStack perspective, it will keep implementation specific concepts from the base data model, and provide a great test case to develop a mechanism to capture this information in 4.3. Ideally, I want CloudStack to exploit these implementation specific features. I just want to provide a facility to manage that data that does not leak across implementations.
Thanks,
-John
On Jun 10, 2013, at 6:07 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> I am pretty sure it is not a blocker for me to drop Burst IOPS. Let me run
> that past Product Management, but I don't think it's a problem.
>
>
> On Mon, Jun 10, 2013 at 4:01 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Yes. I realize my other reply did not explicitly state leaving the Min
>> and Max IOPS fields in the data model as these seem to generic terms across
>> storage implementations.
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> More generally speaking, you're looking to remove Burst IOPS from
>>> CloudStack for 4.2, but we would keep Min and Max (and they would be
>>> displayed in the Disk Offering dialog as I've proposed)?
>>>
>>>
>>> On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Just to make sure I understand your request, are you looking to display
>>>> Min and Max (as long as Wei's feature is not in use), but not display
>> Burst
>>>> IOPS?
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> My concern becomes that we start ballooning the data model and user
>>>>> interface with a fields that are documented as, "If using SolidFire,
>> then
>>>>> Burst IOPS is honored and foo and bar are not. For solution X, Burst
>> IOPS
>>>>> is ignored, but foo and bar apply." It may have to hold until 4.3,
>> but it
>>>>> seems like we need an extended data concept for storage drivers that
>> allow
>>>>> them to define an additional set of properties, and persist them into
>> the
>>>>> database as a JSON document. Such an enhancement would also require
>> some
>>>>> UI fanciness to consume the metadata provided by the driver and adjust
>> the
>>>>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
>>>>> address extended driver data in a holistic manner?
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>>> My thinking is that Min and Max are industry standard and Burst is a
>> new
>>>>>> concept that could catch on.
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>
>>>>>>> Wei,
>>>>>>>
>>>>>>> In this case, we can have the hypervisor or storage providing the
>>>>> quality
>>>>>>> of service guarantee. Naively, it seems reasonable to separate
>>>>> hypervisor
>>>>>>> and storage QoS parameters and enforce mutually exclusion. Not only
>>>>> does
>>>>>>> this approach simplify a whole range of operations, it also avoids
>>>>>>> reconciling the data models. The only question I have about the
>>>>> storage
>>>>>>> provision IOPS is whether or not "Burst IOPS" is an industry standard
>>>>> or
>>>>>>> vendor specific concept.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>> I do not think users can select only one of them, as they are
>>>>> implemented
>>>>>>>> on different sides.
>>>>>>>> Have you investigated the parameters other storage devices support,
>>>>>>> besides
>>>>>>>> min/max/burst IOPS? You'd better add all possible fields in your
>>>>>>>> implementation.
>>>>>>>>
>>>>>>>> What do you think about this?
>>>>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which
>> includes
>>>>> all
>>>>>>>> supported storage vendors.
>>>>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
>>>>>>>> If users select other vendors, relevant fields will appear.
>>>>>>>> Actually I still insist that it is better to add the storage-related
>>>>>>> fields
>>>>>>>> in another table.
>>>>>>>>
>>>>>>>> -Wei
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>
>>>>>>>>> Here is my thinking:
>>>>>>>>>
>>>>>>>>> Two radio buttons (whatever we want to call them):
>>>>>>>>>
>>>>>>>>> 1) Hypervisor IOPS
>>>>>>>>> 2) Storage IOPS
>>>>>>>>>
>>>>>>>>> Leave them both un-checked by default.
>>>>>>>>>
>>>>>>>>> If the user checks one or the other, the relevant fields appear.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> What do you think, Wei?
>>>>>>>>>>
>>>>>>>>>> Should we come up with a way for only one feature (yours or mine)
>>>>> to be
>>>>>>>>>> used at a time on the new Disk Offering dialog?
>>>>>>>>>>
>>>>>>>>>> Since most storage-side provisioned IOPS don't break it down into
>>>>>>>>> separate
>>>>>>>>>> read and write categories, I think that's the way to go (only one
>>>>>>> feature
>>>>>>>>>> or the other).
>>>>>>>>>>
>>>>>>>>>> Any suggestions from a usability standpoint how we want to
>> implement
>>>>>>>>> this?
>>>>>>>>>> It could be as simple as a radio button to turn on your feature
>> and
>>>>>>> mine
>>>>>>>>>> off or vice versa.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jburwell@basho.com
>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> I agree -- I can't image a situation where you would want to use
>>>>> IOPS
>>>>>>>>>>> provisioned by both the hypervisor and storage. There are two
>>>>> points
>>>>>>> of
>>>>>>>>>>> concern -- the UI and the management server. We have to ensure
>>>>> that
>>>>>>> the
>>>>>>>>>>> user can't create a VM from a compute/disk offering combination
>>>>> where
>>>>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage
>>>>>>>>> provisioned
>>>>>>>>>>> IOPS. I think this functional conflict must be resolved in the
>>>>>>>>> management
>>>>>>>>>>> server to ensure that API calls are properly validated with a UX
>>>>> that
>>>>>>>>>>> avoids user confusion. Have Wei and you worked out an approach
>> to
>>>>>>>>>>> resolving this conflict?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Wei has sent me the screen shots.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an
>> issue
>>>>>>>>> here.
>>>>>>>>>>>>
>>>>>>>>>>>> I do support Disk Offerings.
>>>>>>>>>>>>
>>>>>>>>>>>> It looks like Wei has added four new fields to the Disk
>> Offering.
>>>>>>>>>>>>
>>>>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>>>>>>>>
>>>>>>>>>>>> We just need to decide if we should toggle between his and mine.
>>>>>>>>>>>>
>>>>>>>>>>>> I doubt a user would want to use both features at the same time.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Have Wei and you figured out the system level as well (e.g.
>>>>> allowing
>>>>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but
>> no
>>>>>>>>> both)?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's
>>>>> changed in
>>>>>>>>>>> the
>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>> for his feature?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>>>>> jburwell@basho.com
>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>>>>>>>>>>> between a
>>>>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>>>>>>>>> solution
>>>>>>>>>>>>> did
>>>>>>>>>>>>>>> you select?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <ustcweizhou@gmail.com
>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>>>>> Please review the code on
>> https://reviews.apache.org/r/11782
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>> master.
>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>>>>> (local
>>>>>>>>>>> storage
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>>>>> (such as
>>>>>>>>>>>>> iops),
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>>>>>> performance of
>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
>>>>> I/O
>>>>>>> of
>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>> global
>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is included
>>>>> in
>>>>>>>>> the
>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>>> regressions?
>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>>>> restore
>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>>>>>>>>> (CLOUDSTACK-1301
>>>>>>>>>>>>> -
>>>>>>>>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *™*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *™*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *™*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *™*
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *™*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *™*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
I am pretty sure it is not a blocker for me to drop Burst IOPS. Let me run
that past Product Management, but I don't think it's a problem.
On Mon, Jun 10, 2013 at 4:01 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Yes. I realize my other reply did not explicitly state leaving the Min
> and Max IOPS fields in the data model as these seem to generic terms across
> storage implementations.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > More generally speaking, you're looking to remove Burst IOPS from
> > CloudStack for 4.2, but we would keep Min and Max (and they would be
> > displayed in the Disk Offering dialog as I've proposed)?
> >
> >
> > On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> Just to make sure I understand your request, are you looking to display
> >> Min and Max (as long as Wei's feature is not in use), but not display
> Burst
> >> IOPS?
> >>
> >>
> >> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> My concern becomes that we start ballooning the data model and user
> >>> interface with a fields that are documented as, "If using SolidFire,
> then
> >>> Burst IOPS is honored and foo and bar are not. For solution X, Burst
> IOPS
> >>> is ignored, but foo and bar apply." It may have to hold until 4.3,
> but it
> >>> seems like we need an extended data concept for storage drivers that
> allow
> >>> them to define an additional set of properties, and persist them into
> the
> >>> database as a JSON document. Such an enhancement would also require
> some
> >>> UI fanciness to consume the metadata provided by the driver and adjust
> the
> >>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
> >>> address extended driver data in a holistic manner?
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>
> >>>> My thinking is that Min and Max are industry standard and Burst is a
> new
> >>>> concept that could catch on.
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>>>
> >>>>> Wei,
> >>>>>
> >>>>> In this case, we can have the hypervisor or storage providing the
> >>> quality
> >>>>> of service guarantee. Naively, it seems reasonable to separate
> >>> hypervisor
> >>>>> and storage QoS parameters and enforce mutually exclusion. Not only
> >>> does
> >>>>> this approach simplify a whole range of operations, it also avoids
> >>>>> reconciling the data models. The only question I have about the
> >>> storage
> >>>>> provision IOPS is whether or not "Burst IOPS" is an industry standard
> >>> or
> >>>>> vendor specific concept.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>> I do not think users can select only one of them, as they are
> >>> implemented
> >>>>>> on different sides.
> >>>>>> Have you investigated the parameters other storage devices support,
> >>>>> besides
> >>>>>> min/max/burst IOPS? You'd better add all possible fields in your
> >>>>>> implementation.
> >>>>>>
> >>>>>> What do you think about this?
> >>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which
> includes
> >>> all
> >>>>>> supported storage vendors.
> >>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
> >>>>>> If users select other vendors, relevant fields will appear.
> >>>>>> Actually I still insist that it is better to add the storage-related
> >>>>> fields
> >>>>>> in another table.
> >>>>>>
> >>>>>> -Wei
> >>>>>>
> >>>>>>
> >>>>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>
> >>>>>>> Here is my thinking:
> >>>>>>>
> >>>>>>> Two radio buttons (whatever we want to call them):
> >>>>>>>
> >>>>>>> 1) Hypervisor IOPS
> >>>>>>> 2) Storage IOPS
> >>>>>>>
> >>>>>>> Leave them both un-checked by default.
> >>>>>>>
> >>>>>>> If the user checks one or the other, the relevant fields appear.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> What do you think, Wei?
> >>>>>>>>
> >>>>>>>> Should we come up with a way for only one feature (yours or mine)
> >>> to be
> >>>>>>>> used at a time on the new Disk Offering dialog?
> >>>>>>>>
> >>>>>>>> Since most storage-side provisioned IOPS don't break it down into
> >>>>>>> separate
> >>>>>>>> read and write categories, I think that's the way to go (only one
> >>>>> feature
> >>>>>>>> or the other).
> >>>>>>>>
> >>>>>>>> Any suggestions from a usability standpoint how we want to
> implement
> >>>>>>> this?
> >>>>>>>> It could be as simple as a radio button to turn on your feature
> and
> >>>>> mine
> >>>>>>>> off or vice versa.
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jburwell@basho.com
> >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> I agree -- I can't image a situation where you would want to use
> >>> IOPS
> >>>>>>>>> provisioned by both the hypervisor and storage. There are two
> >>> points
> >>>>> of
> >>>>>>>>> concern -- the UI and the management server. We have to ensure
> >>> that
> >>>>> the
> >>>>>>>>> user can't create a VM from a compute/disk offering combination
> >>> where
> >>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage
> >>>>>>> provisioned
> >>>>>>>>> IOPS. I think this functional conflict must be resolved in the
> >>>>>>> management
> >>>>>>>>> server to ensure that API calls are properly validated with a UX
> >>> that
> >>>>>>>>> avoids user confusion. Have Wei and you worked out an approach
> to
> >>>>>>>>> resolving this conflict?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Wei has sent me the screen shots.
> >>>>>>>>>>
> >>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an
> issue
> >>>>>>> here.
> >>>>>>>>>>
> >>>>>>>>>> I do support Disk Offerings.
> >>>>>>>>>>
> >>>>>>>>>> It looks like Wei has added four new fields to the Disk
> Offering.
> >>>>>>>>>>
> >>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
> >>>>>>>>>>
> >>>>>>>>>> We just need to decide if we should toggle between his and mine.
> >>>>>>>>>>
> >>>>>>>>>> I doubt a user would want to use both features at the same time.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
> >>> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> Have Wei and you figured out the system level as well (e.g.
> >>> allowing
> >>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but
> no
> >>>>>>> both)?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's
> >>> changed in
> >>>>>>>>> the
> >>>>>>>>>>> GUI
> >>>>>>>>>>>> for his feature?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
> >>> jburwell@basho.com
> >>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
> >>>>>>>>> between a
> >>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> >>>>>>> solution
> >>>>>>>>>>> did
> >>>>>>>>>>>>> you select?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <ustcweizhou@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>>>>>>>> Please review the code on
> https://reviews.apache.org/r/11782
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
> master.
> >>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Virtual machines are running on the same storage device
> >>> (local
> >>>>>>>>> storage
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
> >>> (such as
> >>>>>>>>>>> iops),
> >>>>>>>>>>>>> if
> >>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>>>>>>> performance of
> >>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
> >>> I/O
> >>>>> of
> >>>>>>>>> VMs.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
> global
> >>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>> JIRA ticket:
> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * What automated testing (unit and integration) is included
> >>> in
> >>>>>>> the
> >>>>>>>>> new
> >>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * What testing has been done to check for potential
> >>> regressions?
> >>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> >>> restore
> >>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> >>>>>>>>>> (CLOUDSTACK-1301
> >>>>>>>>>>> -
> >>>>>>>>>>>>> VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *™*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *™*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *™*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *™*
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *™*
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *™*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Mike,
Yes. I realize my other reply did not explicitly state leaving the Min and Max IOPS fields in the data model as these seem to generic terms across storage implementations.
Thanks,
-John
On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> More generally speaking, you're looking to remove Burst IOPS from
> CloudStack for 4.2, but we would keep Min and Max (and they would be
> displayed in the Disk Offering dialog as I've proposed)?
>
>
> On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Just to make sure I understand your request, are you looking to display
>> Min and Max (as long as Wei's feature is not in use), but not display Burst
>> IOPS?
>>
>>
>> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> My concern becomes that we start ballooning the data model and user
>>> interface with a fields that are documented as, "If using SolidFire, then
>>> Burst IOPS is honored and foo and bar are not. For solution X, Burst IOPS
>>> is ignored, but foo and bar apply." It may have to hold until 4.3, but it
>>> seems like we need an extended data concept for storage drivers that allow
>>> them to define an additional set of properties, and persist them into the
>>> database as a JSON document. Such an enhancement would also require some
>>> UI fanciness to consume the metadata provided by the driver and adjust the
>>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
>>> address extended driver data in a holistic manner?
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>>> My thinking is that Min and Max are industry standard and Burst is a new
>>>> concept that could catch on.
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>
>>>>> Wei,
>>>>>
>>>>> In this case, we can have the hypervisor or storage providing the
>>> quality
>>>>> of service guarantee. Naively, it seems reasonable to separate
>>> hypervisor
>>>>> and storage QoS parameters and enforce mutually exclusion. Not only
>>> does
>>>>> this approach simplify a whole range of operations, it also avoids
>>>>> reconciling the data models. The only question I have about the
>>> storage
>>>>> provision IOPS is whether or not "Burst IOPS" is an industry standard
>>> or
>>>>> vendor specific concept.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>>>
>>>>>> Mike,
>>>>>> I do not think users can select only one of them, as they are
>>> implemented
>>>>>> on different sides.
>>>>>> Have you investigated the parameters other storage devices support,
>>>>> besides
>>>>>> min/max/burst IOPS? You'd better add all possible fields in your
>>>>>> implementation.
>>>>>>
>>>>>> What do you think about this?
>>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which includes
>>> all
>>>>>> supported storage vendors.
>>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
>>>>>> If users select other vendors, relevant fields will appear.
>>>>>> Actually I still insist that it is better to add the storage-related
>>>>> fields
>>>>>> in another table.
>>>>>>
>>>>>> -Wei
>>>>>>
>>>>>>
>>>>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>>>>>>
>>>>>>> Here is my thinking:
>>>>>>>
>>>>>>> Two radio buttons (whatever we want to call them):
>>>>>>>
>>>>>>> 1) Hypervisor IOPS
>>>>>>> 2) Storage IOPS
>>>>>>>
>>>>>>> Leave them both un-checked by default.
>>>>>>>
>>>>>>> If the user checks one or the other, the relevant fields appear.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> What do you think, Wei?
>>>>>>>>
>>>>>>>> Should we come up with a way for only one feature (yours or mine)
>>> to be
>>>>>>>> used at a time on the new Disk Offering dialog?
>>>>>>>>
>>>>>>>> Since most storage-side provisioned IOPS don't break it down into
>>>>>>> separate
>>>>>>>> read and write categories, I think that's the way to go (only one
>>>>> feature
>>>>>>>> or the other).
>>>>>>>>
>>>>>>>> Any suggestions from a usability standpoint how we want to implement
>>>>>>> this?
>>>>>>>> It could be as simple as a radio button to turn on your feature and
>>>>> mine
>>>>>>>> off or vice versa.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> I agree -- I can't image a situation where you would want to use
>>> IOPS
>>>>>>>>> provisioned by both the hypervisor and storage. There are two
>>> points
>>>>> of
>>>>>>>>> concern -- the UI and the management server. We have to ensure
>>> that
>>>>> the
>>>>>>>>> user can't create a VM from a compute/disk offering combination
>>> where
>>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage
>>>>>>> provisioned
>>>>>>>>> IOPS. I think this functional conflict must be resolved in the
>>>>>>> management
>>>>>>>>> server to ensure that API calls are properly validated with a UX
>>> that
>>>>>>>>> avoids user confusion. Have Wei and you worked out an approach to
>>>>>>>>> resolving this conflict?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Wei has sent me the screen shots.
>>>>>>>>>>
>>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an issue
>>>>>>> here.
>>>>>>>>>>
>>>>>>>>>> I do support Disk Offerings.
>>>>>>>>>>
>>>>>>>>>> It looks like Wei has added four new fields to the Disk Offering.
>>>>>>>>>>
>>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>>>>>>
>>>>>>>>>> We just need to decide if we should toggle between his and mine.
>>>>>>>>>>
>>>>>>>>>> I doubt a user would want to use both features at the same time.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
>>> jburwell@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> Have Wei and you figured out the system level as well (e.g.
>>> allowing
>>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but no
>>>>>>> both)?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's
>>> changed in
>>>>>>>>> the
>>>>>>>>>>> GUI
>>>>>>>>>>>> for his feature?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>>> jburwell@basho.com
>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>>>>>>>>> between a
>>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>>>>>>> solution
>>>>>>>>>>> did
>>>>>>>>>>>>> you select?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>>> (local
>>>>>>>>> storage
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>>> (such as
>>>>>>>>>>> iops),
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>>>> performance of
>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
>>> I/O
>>>>> of
>>>>>>>>> VMs.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * What automated testing (unit and integration) is included
>>> in
>>>>>>> the
>>>>>>>>> new
>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>> regressions?
>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>> restore
>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>>>>>>> (CLOUDSTACK-1301
>>>>>>>>>>> -
>>>>>>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *™*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *™*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *™*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *™*
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *™*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
More generally speaking, you're looking to remove Burst IOPS from
CloudStack for 4.2, but we would keep Min and Max (and they would be
displayed in the Disk Offering dialog as I've proposed)?
On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Just to make sure I understand your request, are you looking to display
> Min and Max (as long as Wei's feature is not in use), but not display Burst
> IOPS?
>
>
> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> My concern becomes that we start ballooning the data model and user
>> interface with a fields that are documented as, "If using SolidFire, then
>> Burst IOPS is honored and foo and bar are not. For solution X, Burst IOPS
>> is ignored, but foo and bar apply." It may have to hold until 4.3, but it
>> seems like we need an extended data concept for storage drivers that allow
>> them to define an additional set of properties, and persist them into the
>> database as a JSON document. Such an enhancement would also require some
>> UI fanciness to consume the metadata provided by the driver and adjust the
>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
>> address extended driver data in a holistic manner?
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > My thinking is that Min and Max are industry standard and Burst is a new
>> > concept that could catch on.
>> >
>> >
>> > On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Wei,
>> >>
>> >> In this case, we can have the hypervisor or storage providing the
>> quality
>> >> of service guarantee. Naively, it seems reasonable to separate
>> hypervisor
>> >> and storage QoS parameters and enforce mutually exclusion. Not only
>> does
>> >> this approach simplify a whole range of operations, it also avoids
>> >> reconciling the data models. The only question I have about the
>> storage
>> >> provision IOPS is whether or not "Burst IOPS" is an industry standard
>> or
>> >> vendor specific concept.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>> >>
>> >>> Mike,
>> >>> I do not think users can select only one of them, as they are
>> implemented
>> >>> on different sides.
>> >>> Have you investigated the parameters other storage devices support,
>> >> besides
>> >>> min/max/burst IOPS? You'd better add all possible fields in your
>> >>> implementation.
>> >>>
>> >>> What do you think about this?
>> >>> Hypersivor IOPS is fixed, and there is a drop-down box which includes
>> all
>> >>> supported storage vendors.
>> >>> If users select "SolidFire", min/max/burst IOPS will appear.
>> >>> If users select other vendors, relevant fields will appear.
>> >>> Actually I still insist that it is better to add the storage-related
>> >> fields
>> >>> in another table.
>> >>>
>> >>> -Wei
>> >>>
>> >>>
>> >>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>> >>>
>> >>>> Here is my thinking:
>> >>>>
>> >>>> Two radio buttons (whatever we want to call them):
>> >>>>
>> >>>> 1) Hypervisor IOPS
>> >>>> 2) Storage IOPS
>> >>>>
>> >>>> Leave them both un-checked by default.
>> >>>>
>> >>>> If the user checks one or the other, the relevant fields appear.
>> >>>>
>> >>>>
>> >>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>
>> >>>>> What do you think, Wei?
>> >>>>>
>> >>>>> Should we come up with a way for only one feature (yours or mine)
>> to be
>> >>>>> used at a time on the new Disk Offering dialog?
>> >>>>>
>> >>>>> Since most storage-side provisioned IOPS don't break it down into
>> >>>> separate
>> >>>>> read and write categories, I think that's the way to go (only one
>> >> feature
>> >>>>> or the other).
>> >>>>>
>> >>>>> Any suggestions from a usability standpoint how we want to implement
>> >>>> this?
>> >>>>> It could be as simple as a radio button to turn on your feature and
>> >> mine
>> >>>>> off or vice versa.
>> >>>>>
>> >>>>> Thanks!
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> I agree -- I can't image a situation where you would want to use
>> IOPS
>> >>>>>> provisioned by both the hypervisor and storage. There are two
>> points
>> >> of
>> >>>>>> concern -- the UI and the management server. We have to ensure
>> that
>> >> the
>> >>>>>> user can't create a VM from a compute/disk offering combination
>> where
>> >>>>>> hypervisor throttled I/O would contradict/conflict with storage
>> >>>> provisioned
>> >>>>>> IOPS. I think this functional conflict must be resolved in the
>> >>>> management
>> >>>>>> server to ensure that API calls are properly validated with a UX
>> that
>> >>>>>> avoids user confusion. Have Wei and you worked out an approach to
>> >>>>>> resolving this conflict?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Wei has sent me the screen shots.
>> >>>>>>>
>> >>>>>>> I don't support Compute Offerings for 4.2, so that's not an issue
>> >>>> here.
>> >>>>>>>
>> >>>>>>> I do support Disk Offerings.
>> >>>>>>>
>> >>>>>>> It looks like Wei has added four new fields to the Disk Offering.
>> >>>>>>>
>> >>>>>>> I have added three (Min, Max, and Burst IOPS).
>> >>>>>>>
>> >>>>>>> We just need to decide if we should toggle between his and mine.
>> >>>>>>>
>> >>>>>>> I doubt a user would want to use both features at the same time.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> Have Wei and you figured out the system level as well (e.g.
>> allowing
>> >>>>>>>> either storage provisioned IOPS or hypervisor throttling, but no
>> >>>> both)?
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Perhaps Wei could send me some screen shots of what he's
>> changed in
>> >>>>>> the
>> >>>>>>>> GUI
>> >>>>>>>>> for his feature?
>> >>>>>>>>>
>> >>>>>>>>> Thanks!
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>> jburwell@basho.com
>> >>>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Wei,
>> >>>>>>>>>>
>> >>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>> >>>>>> between a
>> >>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>> >>>> solution
>> >>>>>>>> did
>> >>>>>>>>>> you select?
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>> >>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Guys,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
>> >>>>>>>>>>>
>> >>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>> >>>>>>>>>>>
>> >>>>>>>>>>> -Wei
>> >>>>>>>>>>>
>> >>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi,
>> >>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>> >>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Virtual machines are running on the same storage device
>> (local
>> >>>>>> storage
>> >>>>>>>>>> or
>> >>>>>>>>>>>> share strage). Because of the rate limitation of device
>> (such as
>> >>>>>>>> iops),
>> >>>>>>>>>> if
>> >>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>> >>>>>> performance of
>> >>>>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
>> I/O
>> >> of
>> >>>>>> VMs.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>> >>>>>>>>>>>> configuration)
>> >>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>>>> JIRA ticket:
>> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>> >>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * Did you check the branch's RAT execution success?
>> >>>>>>>>>>>> Yes
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>> No
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * What automated testing (unit and integration) is included
>> in
>> >>>> the
>> >>>>>> new
>> >>>>>>>>>>>> feature?
>> >>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * What testing has been done to check for potential
>> regressions?
>> >>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>> restore
>> >>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>> >>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>> Wei
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> [1]
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>> >>>>>>> (CLOUDSTACK-1301
>> >>>>>>>> -
>> >>>>>>>>>> VM Disk I/O Throttling)
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *™*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *™*
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>> *™*
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the
>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>> *™*
>> >>>>
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Just to make sure I understand your request, are you looking to display Min
and Max (as long as Wei's feature is not in use), but not display Burst
IOPS?
On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> My concern becomes that we start ballooning the data model and user
> interface with a fields that are documented as, "If using SolidFire, then
> Burst IOPS is honored and foo and bar are not. For solution X, Burst IOPS
> is ignored, but foo and bar apply." It may have to hold until 4.3, but it
> seems like we need an extended data concept for storage drivers that allow
> them to define an additional set of properties, and persist them into the
> database as a JSON document. Such an enhancement would also require some
> UI fanciness to consume the metadata provided by the driver and adjust the
> UI. Would it be possible to defer Burst IOPS until 4.3 when we could
> address extended driver data in a holistic manner?
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > My thinking is that Min and Max are industry standard and Burst is a new
> > concept that could catch on.
> >
> >
> > On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Wei,
> >>
> >> In this case, we can have the hypervisor or storage providing the
> quality
> >> of service guarantee. Naively, it seems reasonable to separate
> hypervisor
> >> and storage QoS parameters and enforce mutually exclusion. Not only
> does
> >> this approach simplify a whole range of operations, it also avoids
> >> reconciling the data models. The only question I have about the storage
> >> provision IOPS is whether or not "Burst IOPS" is an industry standard or
> >> vendor specific concept.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
> >>
> >>> Mike,
> >>> I do not think users can select only one of them, as they are
> implemented
> >>> on different sides.
> >>> Have you investigated the parameters other storage devices support,
> >> besides
> >>> min/max/burst IOPS? You'd better add all possible fields in your
> >>> implementation.
> >>>
> >>> What do you think about this?
> >>> Hypersivor IOPS is fixed, and there is a drop-down box which includes
> all
> >>> supported storage vendors.
> >>> If users select "SolidFire", min/max/burst IOPS will appear.
> >>> If users select other vendors, relevant fields will appear.
> >>> Actually I still insist that it is better to add the storage-related
> >> fields
> >>> in another table.
> >>>
> >>> -Wei
> >>>
> >>>
> >>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> >>>
> >>>> Here is my thinking:
> >>>>
> >>>> Two radio buttons (whatever we want to call them):
> >>>>
> >>>> 1) Hypervisor IOPS
> >>>> 2) Storage IOPS
> >>>>
> >>>> Leave them both un-checked by default.
> >>>>
> >>>> If the user checks one or the other, the relevant fields appear.
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> What do you think, Wei?
> >>>>>
> >>>>> Should we come up with a way for only one feature (yours or mine) to
> be
> >>>>> used at a time on the new Disk Offering dialog?
> >>>>>
> >>>>> Since most storage-side provisioned IOPS don't break it down into
> >>>> separate
> >>>>> read and write categories, I think that's the way to go (only one
> >> feature
> >>>>> or the other).
> >>>>>
> >>>>> Any suggestions from a usability standpoint how we want to implement
> >>>> this?
> >>>>> It could be as simple as a radio button to turn on your feature and
> >> mine
> >>>>> off or vice versa.
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> I agree -- I can't image a situation where you would want to use
> IOPS
> >>>>>> provisioned by both the hypervisor and storage. There are two
> points
> >> of
> >>>>>> concern -- the UI and the management server. We have to ensure that
> >> the
> >>>>>> user can't create a VM from a compute/disk offering combination
> where
> >>>>>> hypervisor throttled I/O would contradict/conflict with storage
> >>>> provisioned
> >>>>>> IOPS. I think this functional conflict must be resolved in the
> >>>> management
> >>>>>> server to ensure that API calls are properly validated with a UX
> that
> >>>>>> avoids user confusion. Have Wei and you worked out an approach to
> >>>>>> resolving this conflict?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Wei has sent me the screen shots.
> >>>>>>>
> >>>>>>> I don't support Compute Offerings for 4.2, so that's not an issue
> >>>> here.
> >>>>>>>
> >>>>>>> I do support Disk Offerings.
> >>>>>>>
> >>>>>>> It looks like Wei has added four new fields to the Disk Offering.
> >>>>>>>
> >>>>>>> I have added three (Min, Max, and Burst IOPS).
> >>>>>>>
> >>>>>>> We just need to decide if we should toggle between his and mine.
> >>>>>>>
> >>>>>>> I doubt a user would want to use both features at the same time.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Have Wei and you figured out the system level as well (e.g.
> allowing
> >>>>>>>> either storage provisioned IOPS or hypervisor throttling, but no
> >>>> both)?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Perhaps Wei could send me some screen shots of what he's changed
> in
> >>>>>> the
> >>>>>>>> GUI
> >>>>>>>>> for his feature?
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Wei,
> >>>>>>>>>>
> >>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
> >>>>>> between a
> >>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> >>>> solution
> >>>>>>>> did
> >>>>>>>>>> you select?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
> >>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Guys,
> >>>>>>>>>>>
> >>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
> >>>>>>>>>>>
> >>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
> >>>>>>>>>>>
> >>>>>>>>>>> -Wei
> >>>>>>>>>>>
> >>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>
> >>>>>>>>>>>> Virtual machines are running on the same storage device (local
> >>>>>> storage
> >>>>>>>>>> or
> >>>>>>>>>>>> share strage). Because of the rate limitation of device (such
> as
> >>>>>>>> iops),
> >>>>>>>>>> if
> >>>>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>>>> performance of
> >>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
> >> of
> >>>>>> VMs.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>
> >>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >>>>>>>>>>>> configuration)
> >>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>> JIRA ticket:
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>
> >>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>> Yes
> >>>>>>>>>>>>
> >>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>> No
> >>>>>>>>>>>>
> >>>>>>>>>>>> * What automated testing (unit and integration) is included in
> >>>> the
> >>>>>> new
> >>>>>>>>>>>> feature?
> >>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * What testing has been done to check for potential
> regressions?
> >>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> restore
> >>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>
> >>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best regards,
> >>>>>>>>>>>> Wei
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1]
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> >>>>>>> (CLOUDSTACK-1301
> >>>>>>>> -
> >>>>>>>>>> VM Disk I/O Throttling)
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *™*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *™*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> *™*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *™*
> >>>>
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Mike,
My concern becomes that we start ballooning the data model and user interface with a fields that are documented as, "If using SolidFire, then Burst IOPS is honored and foo and bar are not. For solution X, Burst IOPS is ignored, but foo and bar apply." It may have to hold until 4.3, but it seems like we need an extended data concept for storage drivers that allow them to define an additional set of properties, and persist them into the database as a JSON document. Such an enhancement would also require some UI fanciness to consume the metadata provided by the driver and adjust the UI. Would it be possible to defer Burst IOPS until 4.3 when we could address extended driver data in a holistic manner?
Thanks,
-John
On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> My thinking is that Min and Max are industry standard and Burst is a new
> concept that could catch on.
>
>
> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com> wrote:
>
>> Wei,
>>
>> In this case, we can have the hypervisor or storage providing the quality
>> of service guarantee. Naively, it seems reasonable to separate hypervisor
>> and storage QoS parameters and enforce mutually exclusion. Not only does
>> this approach simplify a whole range of operations, it also avoids
>> reconciling the data models. The only question I have about the storage
>> provision IOPS is whether or not "Burst IOPS" is an industry standard or
>> vendor specific concept.
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Mike,
>>> I do not think users can select only one of them, as they are implemented
>>> on different sides.
>>> Have you investigated the parameters other storage devices support,
>> besides
>>> min/max/burst IOPS? You'd better add all possible fields in your
>>> implementation.
>>>
>>> What do you think about this?
>>> Hypersivor IOPS is fixed, and there is a drop-down box which includes all
>>> supported storage vendors.
>>> If users select "SolidFire", min/max/burst IOPS will appear.
>>> If users select other vendors, relevant fields will appear.
>>> Actually I still insist that it is better to add the storage-related
>> fields
>>> in another table.
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>>>
>>>> Here is my thinking:
>>>>
>>>> Two radio buttons (whatever we want to call them):
>>>>
>>>> 1) Hypervisor IOPS
>>>> 2) Storage IOPS
>>>>
>>>> Leave them both un-checked by default.
>>>>
>>>> If the user checks one or the other, the relevant fields appear.
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> What do you think, Wei?
>>>>>
>>>>> Should we come up with a way for only one feature (yours or mine) to be
>>>>> used at a time on the new Disk Offering dialog?
>>>>>
>>>>> Since most storage-side provisioned IOPS don't break it down into
>>>> separate
>>>>> read and write categories, I think that's the way to go (only one
>> feature
>>>>> or the other).
>>>>>
>>>>> Any suggestions from a usability standpoint how we want to implement
>>>> this?
>>>>> It could be as simple as a radio button to turn on your feature and
>> mine
>>>>> off or vice versa.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> I agree -- I can't image a situation where you would want to use IOPS
>>>>>> provisioned by both the hypervisor and storage. There are two points
>> of
>>>>>> concern -- the UI and the management server. We have to ensure that
>> the
>>>>>> user can't create a VM from a compute/disk offering combination where
>>>>>> hypervisor throttled I/O would contradict/conflict with storage
>>>> provisioned
>>>>>> IOPS. I think this functional conflict must be resolved in the
>>>> management
>>>>>> server to ensure that API calls are properly validated with a UX that
>>>>>> avoids user confusion. Have Wei and you worked out an approach to
>>>>>> resolving this conflict?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Wei has sent me the screen shots.
>>>>>>>
>>>>>>> I don't support Compute Offerings for 4.2, so that's not an issue
>>>> here.
>>>>>>>
>>>>>>> I do support Disk Offerings.
>>>>>>>
>>>>>>> It looks like Wei has added four new fields to the Disk Offering.
>>>>>>>
>>>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>>>
>>>>>>> We just need to decide if we should toggle between his and mine.
>>>>>>>
>>>>>>> I doubt a user would want to use both features at the same time.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Have Wei and you figured out the system level as well (e.g. allowing
>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but no
>>>> both)?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Perhaps Wei could send me some screen shots of what he's changed in
>>>>>> the
>>>>>>>> GUI
>>>>>>>>> for his feature?
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jburwell@basho.com
>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Wei,
>>>>>>>>>>
>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>>>>>> between a
>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>>>> solution
>>>>>>>> did
>>>>>>>>>> you select?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Guys,
>>>>>>>>>>>
>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
>>>>>>>>>>>
>>>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>>>>>>>
>>>>>>>>>>> -Wei
>>>>>>>>>>>
>>>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>
>>>>>>>>>>>> Virtual machines are running on the same storage device (local
>>>>>> storage
>>>>>>>>>> or
>>>>>>>>>>>> share strage). Because of the rate limitation of device (such as
>>>>>>>> iops),
>>>>>>>>>> if
>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>> performance of
>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
>> of
>>>>>> VMs.
>>>>>>>>>>>>
>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>
>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>>>>>>> configuration)
>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>> JIRA ticket:
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>
>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>> Yes
>>>>>>>>>>>>
>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>> No
>>>>>>>>>>>>
>>>>>>>>>>>> * What automated testing (unit and integration) is included in
>>>> the
>>>>>> new
>>>>>>>>>>>> feature?
>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>
>>>>>>>>>>>> * What testing has been done to check for potential regressions?
>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>
>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Wei
>>>>>>>>>>>>
>>>>>>>>>>>> [1]
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>>>> (CLOUDSTACK-1301
>>>>>>>> -
>>>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *™*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *™*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> *™*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *™*
>>>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi,
I don't think we can restrict Burst to SolidFire only because there really
is no way to know that the Disk Offering will be serviced by SolidFire.
This really is a flaw with the way CloudStack handles storage tags.
For example, the Disk Offering could use a storage tag that references
SolidFire and, say, NetApp. For example, say the storage tag is simply
"Fast". Both SolidFire and NetApp could have storage that supports this
requirement to the admin's liking. In this situation, you don't know which
vendor will actually have the volume deployed to it (so you don't know
which subset of fields to have the admin fill out).
I'm thinking we need to have a superset of the fields (Min, Max, and Bust)
and the admin needs to know which vendors can satisfy the Disk Offering
(listed with the storage tags field) and fill in the fields as is
applicable. Let's say another vendor adds a field that SolidFire doesn't
support. It would be available for entry and if the Disk Offering was
deployed to SolidFire, I would ignore that field.
On Mon, Jun 10, 2013 at 2:57 PM, Wei ZHOU <us...@gmail.com> wrote:
> Mike,
>
> Glad to see this. This is what I have questioned about.
>
> I accepted John's and your opnion that hypervisor IOPS and storage IOPS are
> mutually exclusion. The way you mentioned is a good way to go.
>
> A small question is, will burst IOPS only appear when storage device is
> SolidFire?
>
> -Wei
>
>
> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>
> > My thinking is that Min and Max are industry standard and Burst is a new
> > concept that could catch on.
> >
> >
> > On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> > > Wei,
> > >
> > > In this case, we can have the hypervisor or storage providing the
> quality
> > > of service guarantee. Naively, it seems reasonable to separate
> > hypervisor
> > > and storage QoS parameters and enforce mutually exclusion. Not only
> does
> > > this approach simplify a whole range of operations, it also avoids
> > > reconciling the data models. The only question I have about the
> storage
> > > provision IOPS is whether or not "Burst IOPS" is an industry standard
> or
> > > vendor specific concept.
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
> > >
> > > > Mike,
> > > > I do not think users can select only one of them, as they are
> > implemented
> > > > on different sides.
> > > > Have you investigated the parameters other storage devices support,
> > > besides
> > > > min/max/burst IOPS? You'd better add all possible fields in your
> > > > implementation.
> > > >
> > > > What do you think about this?
> > > > Hypersivor IOPS is fixed, and there is a drop-down box which includes
> > all
> > > > supported storage vendors.
> > > > If users select "SolidFire", min/max/burst IOPS will appear.
> > > > If users select other vendors, relevant fields will appear.
> > > > Actually I still insist that it is better to add the storage-related
> > > fields
> > > > in another table.
> > > >
> > > > -Wei
> > > >
> > > >
> > > > 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> > > >
> > > >> Here is my thinking:
> > > >>
> > > >> Two radio buttons (whatever we want to call them):
> > > >>
> > > >> 1) Hypervisor IOPS
> > > >> 2) Storage IOPS
> > > >>
> > > >> Leave them both un-checked by default.
> > > >>
> > > >> If the user checks one or the other, the relevant fields appear.
> > > >>
> > > >>
> > > >> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> > > >> mike.tutkowski@solidfire.com> wrote:
> > > >>
> > > >>> What do you think, Wei?
> > > >>>
> > > >>> Should we come up with a way for only one feature (yours or mine)
> to
> > be
> > > >>> used at a time on the new Disk Offering dialog?
> > > >>>
> > > >>> Since most storage-side provisioned IOPS don't break it down into
> > > >> separate
> > > >>> read and write categories, I think that's the way to go (only one
> > > feature
> > > >>> or the other).
> > > >>>
> > > >>> Any suggestions from a usability standpoint how we want to
> implement
> > > >> this?
> > > >>> It could be as simple as a radio button to turn on your feature and
> > > mine
> > > >>> off or vice versa.
> > > >>>
> > > >>> Thanks!
> > > >>>
> > > >>>
> > > >>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
> > > >> wrote:
> > > >>>
> > > >>>> Mike,
> > > >>>>
> > > >>>> I agree -- I can't image a situation where you would want to use
> > IOPS
> > > >>>> provisioned by both the hypervisor and storage. There are two
> > points
> > > of
> > > >>>> concern -- the UI and the management server. We have to ensure
> that
> > > the
> > > >>>> user can't create a VM from a compute/disk offering combination
> > where
> > > >>>> hypervisor throttled I/O would contradict/conflict with storage
> > > >> provisioned
> > > >>>> IOPS. I think this functional conflict must be resolved in the
> > > >> management
> > > >>>> server to ensure that API calls are properly validated with a UX
> > that
> > > >>>> avoids user confusion. Have Wei and you worked out an approach to
> > > >>>> resolving this conflict?
> > > >>>>
> > > >>>> Thanks,
> > > >>>> -John
> > > >>>>
> > > >>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> > > >> mike.tutkowski@solidfire.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Wei has sent me the screen shots.
> > > >>>>>
> > > >>>>> I don't support Compute Offerings for 4.2, so that's not an issue
> > > >> here.
> > > >>>>>
> > > >>>>> I do support Disk Offerings.
> > > >>>>>
> > > >>>>> It looks like Wei has added four new fields to the Disk Offering.
> > > >>>>>
> > > >>>>> I have added three (Min, Max, and Burst IOPS).
> > > >>>>>
> > > >>>>> We just need to decide if we should toggle between his and mine.
> > > >>>>>
> > > >>>>> I doubt a user would want to use both features at the same time.
> > > >>>>>
> > > >>>>>
> > > >>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <
> jburwell@basho.com
> > >
> > > >>>> wrote:
> > > >>>>>
> > > >>>>>> Mike,
> > > >>>>>>
> > > >>>>>> Have Wei and you figured out the system level as well (e.g.
> > allowing
> > > >>>>>> either storage provisioned IOPS or hypervisor throttling, but no
> > > >> both)?
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> -John
> > > >>>>>>
> > > >>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> > > >>>> mike.tutkowski@solidfire.com>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Perhaps Wei could send me some screen shots of what he's
> changed
> > in
> > > >>>> the
> > > >>>>>> GUI
> > > >>>>>>> for his feature?
> > > >>>>>>>
> > > >>>>>>> Thanks!
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
> > jburwell@basho.com
> > > >
> > > >>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Wei,
> > > >>>>>>>>
> > > >>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
> > > >>>> between a
> > > >>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> > > >> solution
> > > >>>>>> did
> > > >>>>>>>> you select?
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> -John
> > > >>>>>>>>
> > > >>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
> > > >> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Guys,
> > > >>>>>>>>>
> > > >>>>>>>>> I would like to merge disk_io_throttling branch into master.
> > > >>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
> > > >>>>>>>>>
> > > >>>>>>>>> If nobody object, I will merge into master in 72 hours.
> > > >>>>>>>>>
> > > >>>>>>>>> -Wei
> > > >>>>>>>>>
> > > >>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> > > >>>>>>>>>
> > > >>>>>>>>>> Hi,
> > > >>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> > > >>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> > > >>>>>>>>>> The purpose is :
> > > >>>>>>>>>>
> > > >>>>>>>>>> Virtual machines are running on the same storage device
> (local
> > > >>>> storage
> > > >>>>>>>> or
> > > >>>>>>>>>> share strage). Because of the rate limitation of device
> (such
> > as
> > > >>>>>> iops),
> > > >>>>>>>> if
> > > >>>>>>>>>> one VM has large disk operation, it may affect the disk
> > > >>>> performance of
> > > >>>>>>>>>> other VMs running on the same storage device.
> > > >>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
> I/O
> > > of
> > > >>>> VMs.
> > > >>>>>>>>>>
> > > >>>>>>>>>> The feature includes:
> > > >>>>>>>>>>
> > > >>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
> global
> > > >>>>>>>>>> configuration)
> > > >>>>>>>>>> (2) change the maximum rate of VMs
> > > >>>>>>>>>> (3) limit the disk rate (total bps and iops)
> > > >>>>>>>>>> JIRA ticket:
> > > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> > > >>>>>>>>>> FS (I will update later) :
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > > >>>>>>>>>> Merge check list :-
> > > >>>>>>>>>>
> > > >>>>>>>>>> * Did you check the branch's RAT execution success?
> > > >>>>>>>>>> Yes
> > > >>>>>>>>>>
> > > >>>>>>>>>> * Are there new dependencies introduced?
> > > >>>>>>>>>> No
> > > >>>>>>>>>>
> > > >>>>>>>>>> * What automated testing (unit and integration) is included
> in
> > > >> the
> > > >>>> new
> > > >>>>>>>>>> feature?
> > > >>>>>>>>>> Unit tests are added.
> > > >>>>>>>>>>
> > > >>>>>>>>>> * What testing has been done to check for potential
> > regressions?
> > > >>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > > >>>>>>>>>> (2) VM operations, including
> > > >>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> > restore
> > > >>>>>>>>>> (3) Volume operations, including
> > > >>>>>>>>>> Attach, Detach
> > > >>>>>>>>>>
> > > >>>>>>>>>> To review the code, you can try
> > > >>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> > > >>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> > > >>>>>>>>>>
> > > >>>>>>>>>> Best regards,
> > > >>>>>>>>>> Wei
> > > >>>>>>>>>>
> > > >>>>>>>>>> [1]
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > > >>>>>>>>>> [2] refs/heads/disk_io_throttling
> > > >>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> > > >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> > > >>>>> (CLOUDSTACK-1301
> > > >>>>>> -
> > > >>>>>>>> VM Disk I/O Throttling)
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> *Mike Tutkowski*
> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>> o: 303.746.7302
> > > >>>>>>> Advancing the way the world uses the
> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>> *™*
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> *Mike Tutkowski*
> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>> e: mike.tutkowski@solidfire.com
> > > >>>>> o: 303.746.7302
> > > >>>>> Advancing the way the world uses the
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>> *™*
> > > >>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> *Mike Tutkowski*
> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> e: mike.tutkowski@solidfire.com
> > > >>> o: 303.746.7302
> > > >>> Advancing the way the world uses the cloud<
> > > >> http://solidfire.com/solution/overview/?video=play>
> > > >>> *™*
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkowski@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the
> > > >> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >> *™*
> > > >>
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Wei ZHOU <us...@gmail.com>.
Mike,
Glad to see this. This is what I have questioned about.
I accepted John's and your opnion that hypervisor IOPS and storage IOPS are
mutually exclusion. The way you mentioned is a good way to go.
A small question is, will burst IOPS only appear when storage device is
SolidFire?
-Wei
2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> My thinking is that Min and Max are industry standard and Burst is a new
> concept that could catch on.
>
>
> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com> wrote:
>
> > Wei,
> >
> > In this case, we can have the hypervisor or storage providing the quality
> > of service guarantee. Naively, it seems reasonable to separate
> hypervisor
> > and storage QoS parameters and enforce mutually exclusion. Not only does
> > this approach simplify a whole range of operations, it also avoids
> > reconciling the data models. The only question I have about the storage
> > provision IOPS is whether or not "Burst IOPS" is an industry standard or
> > vendor specific concept.
> >
> > Thanks,
> > -John
> >
> > On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
> >
> > > Mike,
> > > I do not think users can select only one of them, as they are
> implemented
> > > on different sides.
> > > Have you investigated the parameters other storage devices support,
> > besides
> > > min/max/burst IOPS? You'd better add all possible fields in your
> > > implementation.
> > >
> > > What do you think about this?
> > > Hypersivor IOPS is fixed, and there is a drop-down box which includes
> all
> > > supported storage vendors.
> > > If users select "SolidFire", min/max/burst IOPS will appear.
> > > If users select other vendors, relevant fields will appear.
> > > Actually I still insist that it is better to add the storage-related
> > fields
> > > in another table.
> > >
> > > -Wei
> > >
> > >
> > > 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> > >
> > >> Here is my thinking:
> > >>
> > >> Two radio buttons (whatever we want to call them):
> > >>
> > >> 1) Hypervisor IOPS
> > >> 2) Storage IOPS
> > >>
> > >> Leave them both un-checked by default.
> > >>
> > >> If the user checks one or the other, the relevant fields appear.
> > >>
> > >>
> > >> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com> wrote:
> > >>
> > >>> What do you think, Wei?
> > >>>
> > >>> Should we come up with a way for only one feature (yours or mine) to
> be
> > >>> used at a time on the new Disk Offering dialog?
> > >>>
> > >>> Since most storage-side provisioned IOPS don't break it down into
> > >> separate
> > >>> read and write categories, I think that's the way to go (only one
> > feature
> > >>> or the other).
> > >>>
> > >>> Any suggestions from a usability standpoint how we want to implement
> > >> this?
> > >>> It could be as simple as a radio button to turn on your feature and
> > mine
> > >>> off or vice versa.
> > >>>
> > >>> Thanks!
> > >>>
> > >>>
> > >>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
> > >> wrote:
> > >>>
> > >>>> Mike,
> > >>>>
> > >>>> I agree -- I can't image a situation where you would want to use
> IOPS
> > >>>> provisioned by both the hypervisor and storage. There are two
> points
> > of
> > >>>> concern -- the UI and the management server. We have to ensure that
> > the
> > >>>> user can't create a VM from a compute/disk offering combination
> where
> > >>>> hypervisor throttled I/O would contradict/conflict with storage
> > >> provisioned
> > >>>> IOPS. I think this functional conflict must be resolved in the
> > >> management
> > >>>> server to ensure that API calls are properly validated with a UX
> that
> > >>>> avoids user confusion. Have Wei and you worked out an approach to
> > >>>> resolving this conflict?
> > >>>>
> > >>>> Thanks,
> > >>>> -John
> > >>>>
> > >>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Wei has sent me the screen shots.
> > >>>>>
> > >>>>> I don't support Compute Offerings for 4.2, so that's not an issue
> > >> here.
> > >>>>>
> > >>>>> I do support Disk Offerings.
> > >>>>>
> > >>>>> It looks like Wei has added four new fields to the Disk Offering.
> > >>>>>
> > >>>>> I have added three (Min, Max, and Burst IOPS).
> > >>>>>
> > >>>>> We just need to decide if we should toggle between his and mine.
> > >>>>>
> > >>>>> I doubt a user would want to use both features at the same time.
> > >>>>>
> > >>>>>
> > >>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jburwell@basho.com
> >
> > >>>> wrote:
> > >>>>>
> > >>>>>> Mike,
> > >>>>>>
> > >>>>>> Have Wei and you figured out the system level as well (e.g.
> allowing
> > >>>>>> either storage provisioned IOPS or hypervisor throttling, but no
> > >> both)?
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> -John
> > >>>>>>
> > >>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> > >>>> mike.tutkowski@solidfire.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Perhaps Wei could send me some screen shots of what he's changed
> in
> > >>>> the
> > >>>>>> GUI
> > >>>>>>> for his feature?
> > >>>>>>>
> > >>>>>>> Thanks!
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
> jburwell@basho.com
> > >
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Wei,
> > >>>>>>>>
> > >>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
> > >>>> between a
> > >>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> > >> solution
> > >>>>>> did
> > >>>>>>>> you select?
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> -John
> > >>>>>>>>
> > >>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
> > >> wrote:
> > >>>>>>>>
> > >>>>>>>>> Guys,
> > >>>>>>>>>
> > >>>>>>>>> I would like to merge disk_io_throttling branch into master.
> > >>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
> > >>>>>>>>>
> > >>>>>>>>> If nobody object, I will merge into master in 72 hours.
> > >>>>>>>>>
> > >>>>>>>>> -Wei
> > >>>>>>>>>
> > >>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> > >>>>>>>>>
> > >>>>>>>>>> Hi,
> > >>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> > >>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> > >>>>>>>>>> The purpose is :
> > >>>>>>>>>>
> > >>>>>>>>>> Virtual machines are running on the same storage device (local
> > >>>> storage
> > >>>>>>>> or
> > >>>>>>>>>> share strage). Because of the rate limitation of device (such
> as
> > >>>>>> iops),
> > >>>>>>>> if
> > >>>>>>>>>> one VM has large disk operation, it may affect the disk
> > >>>> performance of
> > >>>>>>>>>> other VMs running on the same storage device.
> > >>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
> > of
> > >>>> VMs.
> > >>>>>>>>>>
> > >>>>>>>>>> The feature includes:
> > >>>>>>>>>>
> > >>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> > >>>>>>>>>> configuration)
> > >>>>>>>>>> (2) change the maximum rate of VMs
> > >>>>>>>>>> (3) limit the disk rate (total bps and iops)
> > >>>>>>>>>> JIRA ticket:
> > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> > >>>>>>>>>> FS (I will update later) :
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >>>>>>>>>> Merge check list :-
> > >>>>>>>>>>
> > >>>>>>>>>> * Did you check the branch's RAT execution success?
> > >>>>>>>>>> Yes
> > >>>>>>>>>>
> > >>>>>>>>>> * Are there new dependencies introduced?
> > >>>>>>>>>> No
> > >>>>>>>>>>
> > >>>>>>>>>> * What automated testing (unit and integration) is included in
> > >> the
> > >>>> new
> > >>>>>>>>>> feature?
> > >>>>>>>>>> Unit tests are added.
> > >>>>>>>>>>
> > >>>>>>>>>> * What testing has been done to check for potential
> regressions?
> > >>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > >>>>>>>>>> (2) VM operations, including
> > >>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> restore
> > >>>>>>>>>> (3) Volume operations, including
> > >>>>>>>>>> Attach, Detach
> > >>>>>>>>>>
> > >>>>>>>>>> To review the code, you can try
> > >>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> > >>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> > >>>>>>>>>>
> > >>>>>>>>>> Best regards,
> > >>>>>>>>>> Wei
> > >>>>>>>>>>
> > >>>>>>>>>> [1]
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >>>>>>>>>> [2] refs/heads/disk_io_throttling
> > >>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> > >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> > >>>>> (CLOUDSTACK-1301
> > >>>>>> -
> > >>>>>>>> VM Disk I/O Throttling)
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> *Mike Tutkowski*
> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>> o: 303.746.7302
> > >>>>>>> Advancing the way the world uses the
> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>> *™*
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> *Mike Tutkowski*
> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>> e: mike.tutkowski@solidfire.com
> > >>>>> o: 303.746.7302
> > >>>>> Advancing the way the world uses the
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>> *™*
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> *Mike Tutkowski*
> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>> e: mike.tutkowski@solidfire.com
> > >>> o: 303.746.7302
> > >>> Advancing the way the world uses the cloud<
> > >> http://solidfire.com/solution/overview/?video=play>
> > >>> *™*
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> *Mike Tutkowski*
> > >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> e: mike.tutkowski@solidfire.com
> > >> o: 303.746.7302
> > >> Advancing the way the world uses the
> > >> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *™*
> > >>
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
A major issue for current QoS providers is not being able to utilize over a
Max amount even when it is highly desirable and the storage system can
support it. That's why I'm thinking it will be a feature others attempt to
imitate.
On Mon, Jun 10, 2013 at 2:44 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> My thinking is that Min and Max are industry standard and Burst is a new
> concept that could catch on.
>
>
> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com> wrote:
>
>> Wei,
>>
>> In this case, we can have the hypervisor or storage providing the quality
>> of service guarantee. Naively, it seems reasonable to separate hypervisor
>> and storage QoS parameters and enforce mutually exclusion. Not only does
>> this approach simplify a whole range of operations, it also avoids
>> reconciling the data models. The only question I have about the storage
>> provision IOPS is whether or not "Burst IOPS" is an industry standard or
>> vendor specific concept.
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>> > Mike,
>> > I do not think users can select only one of them, as they are
>> implemented
>> > on different sides.
>> > Have you investigated the parameters other storage devices support,
>> besides
>> > min/max/burst IOPS? You'd better add all possible fields in your
>> > implementation.
>> >
>> > What do you think about this?
>> > Hypersivor IOPS is fixed, and there is a drop-down box which includes
>> all
>> > supported storage vendors.
>> > If users select "SolidFire", min/max/burst IOPS will appear.
>> > If users select other vendors, relevant fields will appear.
>> > Actually I still insist that it is better to add the storage-related
>> fields
>> > in another table.
>> >
>> > -Wei
>> >
>> >
>> > 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>> >
>> >> Here is my thinking:
>> >>
>> >> Two radio buttons (whatever we want to call them):
>> >>
>> >> 1) Hypervisor IOPS
>> >> 2) Storage IOPS
>> >>
>> >> Leave them both un-checked by default.
>> >>
>> >> If the user checks one or the other, the relevant fields appear.
>> >>
>> >>
>> >> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com> wrote:
>> >>
>> >>> What do you think, Wei?
>> >>>
>> >>> Should we come up with a way for only one feature (yours or mine) to
>> be
>> >>> used at a time on the new Disk Offering dialog?
>> >>>
>> >>> Since most storage-side provisioned IOPS don't break it down into
>> >> separate
>> >>> read and write categories, I think that's the way to go (only one
>> feature
>> >>> or the other).
>> >>>
>> >>> Any suggestions from a usability standpoint how we want to implement
>> >> this?
>> >>> It could be as simple as a radio button to turn on your feature and
>> mine
>> >>> off or vice versa.
>> >>>
>> >>> Thanks!
>> >>>
>> >>>
>> >>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> I agree -- I can't image a situation where you would want to use IOPS
>> >>>> provisioned by both the hypervisor and storage. There are two
>> points of
>> >>>> concern -- the UI and the management server. We have to ensure that
>> the
>> >>>> user can't create a VM from a compute/disk offering combination where
>> >>>> hypervisor throttled I/O would contradict/conflict with storage
>> >> provisioned
>> >>>> IOPS. I think this functional conflict must be resolved in the
>> >> management
>> >>>> server to ensure that API calls are properly validated with a UX that
>> >>>> avoids user confusion. Have Wei and you worked out an approach to
>> >>>> resolving this conflict?
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com>
>> >>>> wrote:
>> >>>>
>> >>>>> Wei has sent me the screen shots.
>> >>>>>
>> >>>>> I don't support Compute Offerings for 4.2, so that's not an issue
>> >> here.
>> >>>>>
>> >>>>> I do support Disk Offerings.
>> >>>>>
>> >>>>> It looks like Wei has added four new fields to the Disk Offering.
>> >>>>>
>> >>>>> I have added three (Min, Max, and Burst IOPS).
>> >>>>>
>> >>>>> We just need to decide if we should toggle between his and mine.
>> >>>>>
>> >>>>> I doubt a user would want to use both features at the same time.
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> Have Wei and you figured out the system level as well (e.g.
>> allowing
>> >>>>>> either storage provisioned IOPS or hypervisor throttling, but no
>> >> both)?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Perhaps Wei could send me some screen shots of what he's changed
>> in
>> >>>> the
>> >>>>>> GUI
>> >>>>>>> for his feature?
>> >>>>>>>
>> >>>>>>> Thanks!
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <
>> jburwell@basho.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Wei,
>> >>>>>>>>
>> >>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>> >>>> between a
>> >>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>> >> solution
>> >>>>>> did
>> >>>>>>>> you select?
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>> >> wrote:
>> >>>>>>>>
>> >>>>>>>>> Guys,
>> >>>>>>>>>
>> >>>>>>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
>> >>>>>>>>>
>> >>>>>>>>> If nobody object, I will merge into master in 72 hours.
>> >>>>>>>>>
>> >>>>>>>>> -Wei
>> >>>>>>>>>
>> >>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>> >>>>>>>>>
>> >>>>>>>>>> Hi,
>> >>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>> >>>>>>>>>> The purpose is :
>> >>>>>>>>>>
>> >>>>>>>>>> Virtual machines are running on the same storage device (local
>> >>>> storage
>> >>>>>>>> or
>> >>>>>>>>>> share strage). Because of the rate limitation of device (such
>> as
>> >>>>>> iops),
>> >>>>>>>> if
>> >>>>>>>>>> one VM has large disk operation, it may affect the disk
>> >>>> performance of
>> >>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
>> of
>> >>>> VMs.
>> >>>>>>>>>>
>> >>>>>>>>>> The feature includes:
>> >>>>>>>>>>
>> >>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>> >>>>>>>>>> configuration)
>> >>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>> JIRA ticket:
>> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>> >>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>> Merge check list :-
>> >>>>>>>>>>
>> >>>>>>>>>> * Did you check the branch's RAT execution success?
>> >>>>>>>>>> Yes
>> >>>>>>>>>>
>> >>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>> No
>> >>>>>>>>>>
>> >>>>>>>>>> * What automated testing (unit and integration) is included in
>> >> the
>> >>>> new
>> >>>>>>>>>> feature?
>> >>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>
>> >>>>>>>>>> * What testing has been done to check for potential
>> regressions?
>> >>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>> >>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>> Attach, Detach
>> >>>>>>>>>>
>> >>>>>>>>>> To review the code, you can try
>> >>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>> >>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>> >>>>>>>>>>
>> >>>>>>>>>> Best regards,
>> >>>>>>>>>> Wei
>> >>>>>>>>>>
>> >>>>>>>>>> [1]
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>> >>>>> (CLOUDSTACK-1301
>> >>>>>> -
>> >>>>>>>> VM Disk I/O Throttling)
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *™*
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *™*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>> *™*
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the
>> >> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> *™*
>> >>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
My thinking is that Min and Max are industry standard and Burst is a new
concept that could catch on.
On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jb...@basho.com> wrote:
> Wei,
>
> In this case, we can have the hypervisor or storage providing the quality
> of service guarantee. Naively, it seems reasonable to separate hypervisor
> and storage QoS parameters and enforce mutually exclusion. Not only does
> this approach simplify a whole range of operations, it also avoids
> reconciling the data models. The only question I have about the storage
> provision IOPS is whether or not "Burst IOPS" is an industry standard or
> vendor specific concept.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
>
> > Mike,
> > I do not think users can select only one of them, as they are implemented
> > on different sides.
> > Have you investigated the parameters other storage devices support,
> besides
> > min/max/burst IOPS? You'd better add all possible fields in your
> > implementation.
> >
> > What do you think about this?
> > Hypersivor IOPS is fixed, and there is a drop-down box which includes all
> > supported storage vendors.
> > If users select "SolidFire", min/max/burst IOPS will appear.
> > If users select other vendors, relevant fields will appear.
> > Actually I still insist that it is better to add the storage-related
> fields
> > in another table.
> >
> > -Wei
> >
> >
> > 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> >
> >> Here is my thinking:
> >>
> >> Two radio buttons (whatever we want to call them):
> >>
> >> 1) Hypervisor IOPS
> >> 2) Storage IOPS
> >>
> >> Leave them both un-checked by default.
> >>
> >> If the user checks one or the other, the relevant fields appear.
> >>
> >>
> >> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> What do you think, Wei?
> >>>
> >>> Should we come up with a way for only one feature (yours or mine) to be
> >>> used at a time on the new Disk Offering dialog?
> >>>
> >>> Since most storage-side provisioned IOPS don't break it down into
> >> separate
> >>> read and write categories, I think that's the way to go (only one
> feature
> >>> or the other).
> >>>
> >>> Any suggestions from a usability standpoint how we want to implement
> >> this?
> >>> It could be as simple as a radio button to turn on your feature and
> mine
> >>> off or vice versa.
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I agree -- I can't image a situation where you would want to use IOPS
> >>>> provisioned by both the hypervisor and storage. There are two points
> of
> >>>> concern -- the UI and the management server. We have to ensure that
> the
> >>>> user can't create a VM from a compute/disk offering combination where
> >>>> hypervisor throttled I/O would contradict/conflict with storage
> >> provisioned
> >>>> IOPS. I think this functional conflict must be resolved in the
> >> management
> >>>> server to ensure that API calls are properly validated with a UX that
> >>>> avoids user confusion. Have Wei and you worked out an approach to
> >>>> resolving this conflict?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Wei has sent me the screen shots.
> >>>>>
> >>>>> I don't support Compute Offerings for 4.2, so that's not an issue
> >> here.
> >>>>>
> >>>>> I do support Disk Offerings.
> >>>>>
> >>>>> It looks like Wei has added four new fields to the Disk Offering.
> >>>>>
> >>>>> I have added three (Min, Max, and Burst IOPS).
> >>>>>
> >>>>> We just need to decide if we should toggle between his and mine.
> >>>>>
> >>>>> I doubt a user would want to use both features at the same time.
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Have Wei and you figured out the system level as well (e.g. allowing
> >>>>>> either storage provisioned IOPS or hypervisor throttling, but no
> >> both)?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Perhaps Wei could send me some screen shots of what he's changed in
> >>>> the
> >>>>>> GUI
> >>>>>>> for his feature?
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Wei,
> >>>>>>>>
> >>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
> >>>> between a
> >>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> >> solution
> >>>>>> did
> >>>>>>>> you select?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
> >> wrote:
> >>>>>>>>
> >>>>>>>>> Guys,
> >>>>>>>>>
> >>>>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
> >>>>>>>>>
> >>>>>>>>> If nobody object, I will merge into master in 72 hours.
> >>>>>>>>>
> >>>>>>>>> -Wei
> >>>>>>>>>
> >>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>> The purpose is :
> >>>>>>>>>>
> >>>>>>>>>> Virtual machines are running on the same storage device (local
> >>>> storage
> >>>>>>>> or
> >>>>>>>>>> share strage). Because of the rate limitation of device (such as
> >>>>>> iops),
> >>>>>>>> if
> >>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>> performance of
> >>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
> of
> >>>> VMs.
> >>>>>>>>>>
> >>>>>>>>>> The feature includes:
> >>>>>>>>>>
> >>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >>>>>>>>>> configuration)
> >>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>> JIRA ticket:
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>> Merge check list :-
> >>>>>>>>>>
> >>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>> Yes
> >>>>>>>>>>
> >>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>> No
> >>>>>>>>>>
> >>>>>>>>>> * What automated testing (unit and integration) is included in
> >> the
> >>>> new
> >>>>>>>>>> feature?
> >>>>>>>>>> Unit tests are added.
> >>>>>>>>>>
> >>>>>>>>>> * What testing has been done to check for potential regressions?
> >>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>> (2) VM operations, including
> >>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>> Attach, Detach
> >>>>>>>>>>
> >>>>>>>>>> To review the code, you can try
> >>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >>>>>>>>>>
> >>>>>>>>>> Best regards,
> >>>>>>>>>> Wei
> >>>>>>>>>>
> >>>>>>>>>> [1]
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> >>>>> (CLOUDSTACK-1301
> >>>>>> -
> >>>>>>>> VM Disk I/O Throttling)
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *™*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *™*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>> *™*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *™*
> >>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Wei,
In this case, we can have the hypervisor or storage providing the quality of service guarantee. Naively, it seems reasonable to separate hypervisor and storage QoS parameters and enforce mutually exclusion. Not only does this approach simplify a whole range of operations, it also avoids reconciling the data models. The only question I have about the storage provision IOPS is whether or not "Burst IOPS" is an industry standard or vendor specific concept.
Thanks,
-John
On Jun 10, 2013, at 3:59 PM, Wei ZHOU <us...@gmail.com> wrote:
> Mike,
> I do not think users can select only one of them, as they are implemented
> on different sides.
> Have you investigated the parameters other storage devices support, besides
> min/max/burst IOPS? You'd better add all possible fields in your
> implementation.
>
> What do you think about this?
> Hypersivor IOPS is fixed, and there is a drop-down box which includes all
> supported storage vendors.
> If users select "SolidFire", min/max/burst IOPS will appear.
> If users select other vendors, relevant fields will appear.
> Actually I still insist that it is better to add the storage-related fields
> in another table.
>
> -Wei
>
>
> 2013/6/10 Mike Tutkowski <mi...@solidfire.com>
>
>> Here is my thinking:
>>
>> Two radio buttons (whatever we want to call them):
>>
>> 1) Hypervisor IOPS
>> 2) Storage IOPS
>>
>> Leave them both un-checked by default.
>>
>> If the user checks one or the other, the relevant fields appear.
>>
>>
>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> What do you think, Wei?
>>>
>>> Should we come up with a way for only one feature (yours or mine) to be
>>> used at a time on the new Disk Offering dialog?
>>>
>>> Since most storage-side provisioned IOPS don't break it down into
>> separate
>>> read and write categories, I think that's the way to go (only one feature
>>> or the other).
>>>
>>> Any suggestions from a usability standpoint how we want to implement
>> this?
>>> It could be as simple as a radio button to turn on your feature and mine
>>> off or vice versa.
>>>
>>> Thanks!
>>>
>>>
>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> I agree -- I can't image a situation where you would want to use IOPS
>>>> provisioned by both the hypervisor and storage. There are two points of
>>>> concern -- the UI and the management server. We have to ensure that the
>>>> user can't create a VM from a compute/disk offering combination where
>>>> hypervisor throttled I/O would contradict/conflict with storage
>> provisioned
>>>> IOPS. I think this functional conflict must be resolved in the
>> management
>>>> server to ensure that API calls are properly validated with a UX that
>>>> avoids user confusion. Have Wei and you worked out an approach to
>>>> resolving this conflict?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> Wei has sent me the screen shots.
>>>>>
>>>>> I don't support Compute Offerings for 4.2, so that's not an issue
>> here.
>>>>>
>>>>> I do support Disk Offerings.
>>>>>
>>>>> It looks like Wei has added four new fields to the Disk Offering.
>>>>>
>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>
>>>>> We just need to decide if we should toggle between his and mine.
>>>>>
>>>>> I doubt a user would want to use both features at the same time.
>>>>>
>>>>>
>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Have Wei and you figured out the system level as well (e.g. allowing
>>>>>> either storage provisioned IOPS or hypervisor throttling, but no
>> both)?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Perhaps Wei could send me some screen shots of what he's changed in
>>>> the
>>>>>> GUI
>>>>>>> for his feature?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Wei,
>>>>>>>>
>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict
>>>> between a
>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what
>> solution
>>>>>> did
>>>>>>>> you select?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
>> wrote:
>>>>>>>>
>>>>>>>>> Guys,
>>>>>>>>>
>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>> Please review the code on https://reviews.apache.org/r/11782
>>>>>>>>>
>>>>>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>>>>>
>>>>>>>>> -Wei
>>>>>>>>>
>>>>>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>> The purpose is :
>>>>>>>>>>
>>>>>>>>>> Virtual machines are running on the same storage device (local
>>>> storage
>>>>>>>> or
>>>>>>>>>> share strage). Because of the rate limitation of device (such as
>>>>>> iops),
>>>>>>>> if
>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>> performance of
>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>> VMs.
>>>>>>>>>>
>>>>>>>>>> The feature includes:
>>>>>>>>>>
>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>>>>> configuration)
>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>> JIRA ticket:
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>> Merge check list :-
>>>>>>>>>>
>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>> Yes
>>>>>>>>>>
>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>> No
>>>>>>>>>>
>>>>>>>>>> * What automated testing (unit and integration) is included in
>> the
>>>> new
>>>>>>>>>> feature?
>>>>>>>>>> Unit tests are added.
>>>>>>>>>>
>>>>>>>>>> * What testing has been done to check for potential regressions?
>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>> (2) VM operations, including
>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>> Attach, Detach
>>>>>>>>>>
>>>>>>>>>> To review the code, you can try
>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Wei
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>> (CLOUDSTACK-1301
>>>>>> -
>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *™*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *™*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>> *™*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*
>>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Wei ZHOU <us...@gmail.com>.
Mike,
I do not think users can select only one of them, as they are implemented
on different sides.
Have you investigated the parameters other storage devices support, besides
min/max/burst IOPS? You'd better add all possible fields in your
implementation.
What do you think about this?
Hypersivor IOPS is fixed, and there is a drop-down box which includes all
supported storage vendors.
If users select "SolidFire", min/max/burst IOPS will appear.
If users select other vendors, relevant fields will appear.
Actually I still insist that it is better to add the storage-related fields
in another table.
-Wei
2013/6/10 Mike Tutkowski <mi...@solidfire.com>
> Here is my thinking:
>
> Two radio buttons (whatever we want to call them):
>
> 1) Hypervisor IOPS
> 2) Storage IOPS
>
> Leave them both un-checked by default.
>
> If the user checks one or the other, the relevant fields appear.
>
>
> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
> > What do you think, Wei?
> >
> > Should we come up with a way for only one feature (yours or mine) to be
> > used at a time on the new Disk Offering dialog?
> >
> > Since most storage-side provisioned IOPS don't break it down into
> separate
> > read and write categories, I think that's the way to go (only one feature
> > or the other).
> >
> > Any suggestions from a usability standpoint how we want to implement
> this?
> > It could be as simple as a radio button to turn on your feature and mine
> > off or vice versa.
> >
> > Thanks!
> >
> >
> > On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> I agree -- I can't image a situation where you would want to use IOPS
> >> provisioned by both the hypervisor and storage. There are two points of
> >> concern -- the UI and the management server. We have to ensure that the
> >> user can't create a VM from a compute/disk offering combination where
> >> hypervisor throttled I/O would contradict/conflict with storage
> provisioned
> >> IOPS. I think this functional conflict must be resolved in the
> management
> >> server to ensure that API calls are properly validated with a UX that
> >> avoids user confusion. Have Wei and you worked out an approach to
> >> resolving this conflict?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >> > Wei has sent me the screen shots.
> >> >
> >> > I don't support Compute Offerings for 4.2, so that's not an issue
> here.
> >> >
> >> > I do support Disk Offerings.
> >> >
> >> > It looks like Wei has added four new fields to the Disk Offering.
> >> >
> >> > I have added three (Min, Max, and Burst IOPS).
> >> >
> >> > We just need to decide if we should toggle between his and mine.
> >> >
> >> > I doubt a user would want to use both features at the same time.
> >> >
> >> >
> >> > On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >> >
> >> >> Mike,
> >> >>
> >> >> Have Wei and you figured out the system level as well (e.g. allowing
> >> >> either storage provisioned IOPS or hypervisor throttling, but no
> both)?
> >> >>
> >> >> Thanks,
> >> >> -John
> >> >>
> >> >> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >> >> wrote:
> >> >>
> >> >>> Perhaps Wei could send me some screen shots of what he's changed in
> >> the
> >> >> GUI
> >> >>> for his feature?
> >> >>>
> >> >>> Thanks!
> >> >>>
> >> >>>
> >> >>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com>
> >> >> wrote:
> >> >>>
> >> >>>> Wei,
> >> >>>>
> >> >>>> Have Mike Tutkowski and you reconciled the potential conflict
> >> between a
> >> >>>> throttled I/O VM and a provisioned IOPs volume? If so, what
> solution
> >> >> did
> >> >>>> you select?
> >> >>>>
> >> >>>> Thanks,
> >> >>>> -John
> >> >>>>
> >> >>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com>
> wrote:
> >> >>>>
> >> >>>>> Guys,
> >> >>>>>
> >> >>>>> I would like to merge disk_io_throttling branch into master.
> >> >>>>> Please review the code on https://reviews.apache.org/r/11782
> >> >>>>>
> >> >>>>> If nobody object, I will merge into master in 72 hours.
> >> >>>>>
> >> >>>>> -Wei
> >> >>>>>
> >> >>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >> >>>>>
> >> >>>>>> Hi,
> >> >>>>>> I would like to merge disk_io_throttling branch into master.
> >> >>>>>> If nobody object, I will merge into master in 48 hours.
> >> >>>>>> The purpose is :
> >> >>>>>>
> >> >>>>>> Virtual machines are running on the same storage device (local
> >> storage
> >> >>>> or
> >> >>>>>> share strage). Because of the rate limitation of device (such as
> >> >> iops),
> >> >>>> if
> >> >>>>>> one VM has large disk operation, it may affect the disk
> >> performance of
> >> >>>>>> other VMs running on the same storage device.
> >> >>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
> >> VMs.
> >> >>>>>>
> >> >>>>>> The feature includes:
> >> >>>>>>
> >> >>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >> >>>>>> configuration)
> >> >>>>>> (2) change the maximum rate of VMs
> >> >>>>>> (3) limit the disk rate (total bps and iops)
> >> >>>>>> JIRA ticket:
> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >> >>>>>> FS (I will update later) :
> >> >>>>>>
> >> >>>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >> >>>>>> Merge check list :-
> >> >>>>>>
> >> >>>>>> * Did you check the branch's RAT execution success?
> >> >>>>>> Yes
> >> >>>>>>
> >> >>>>>> * Are there new dependencies introduced?
> >> >>>>>> No
> >> >>>>>>
> >> >>>>>> * What automated testing (unit and integration) is included in
> the
> >> new
> >> >>>>>> feature?
> >> >>>>>> Unit tests are added.
> >> >>>>>>
> >> >>>>>> * What testing has been done to check for potential regressions?
> >> >>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >> >>>>>> (2) VM operations, including
> >> >>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >> >>>>>> (3) Volume operations, including
> >> >>>>>> Attach, Detach
> >> >>>>>>
> >> >>>>>> To review the code, you can try
> >> >>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >> >>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >> >>>>>>
> >> >>>>>> Best regards,
> >> >>>>>> Wei
> >> >>>>>>
> >> >>>>>> [1]
> >> >>>>>>
> >> >>>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >> >>>>>> [2] refs/heads/disk_io_throttling
> >> >>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> >> >(CLOUDSTACK-1301
> >> >> -
> >> >>>> VM Disk I/O Throttling)
> >> >>>>>>
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> *Mike Tutkowski*
> >> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >> >>> e: mike.tutkowski@solidfire.com
> >> >>> o: 303.746.7302
> >> >>> Advancing the way the world uses the
> >> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> >>> *™*
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *™*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Here is my thinking:
Two radio buttons (whatever we want to call them):
1) Hypervisor IOPS
2) Storage IOPS
Leave them both un-checked by default.
If the user checks one or the other, the relevant fields appear.
On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> What do you think, Wei?
>
> Should we come up with a way for only one feature (yours or mine) to be
> used at a time on the new Disk Offering dialog?
>
> Since most storage-side provisioned IOPS don't break it down into separate
> read and write categories, I think that's the way to go (only one feature
> or the other).
>
> Any suggestions from a usability standpoint how we want to implement this?
> It could be as simple as a radio button to turn on your feature and mine
> off or vice versa.
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I agree -- I can't image a situation where you would want to use IOPS
>> provisioned by both the hypervisor and storage. There are two points of
>> concern -- the UI and the management server. We have to ensure that the
>> user can't create a VM from a compute/disk offering combination where
>> hypervisor throttled I/O would contradict/conflict with storage provisioned
>> IOPS. I think this functional conflict must be resolved in the management
>> server to ensure that API calls are properly validated with a UX that
>> avoids user confusion. Have Wei and you worked out an approach to
>> resolving this conflict?
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Wei has sent me the screen shots.
>> >
>> > I don't support Compute Offerings for 4.2, so that's not an issue here.
>> >
>> > I do support Disk Offerings.
>> >
>> > It looks like Wei has added four new fields to the Disk Offering.
>> >
>> > I have added three (Min, Max, and Burst IOPS).
>> >
>> > We just need to decide if we should toggle between his and mine.
>> >
>> > I doubt a user would want to use both features at the same time.
>> >
>> >
>> > On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Mike,
>> >>
>> >> Have Wei and you figured out the system level as well (e.g. allowing
>> >> either storage provisioned IOPS or hypervisor throttling, but no both)?
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >> wrote:
>> >>
>> >>> Perhaps Wei could send me some screen shots of what he's changed in
>> the
>> >> GUI
>> >>> for his feature?
>> >>>
>> >>> Thanks!
>> >>>
>> >>>
>> >>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>
>> >>>> Wei,
>> >>>>
>> >>>> Have Mike Tutkowski and you reconciled the potential conflict
>> between a
>> >>>> throttled I/O VM and a provisioned IOPs volume? If so, what solution
>> >> did
>> >>>> you select?
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
>> >>>>
>> >>>>> Guys,
>> >>>>>
>> >>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>> Please review the code on https://reviews.apache.org/r/11782
>> >>>>>
>> >>>>> If nobody object, I will merge into master in 72 hours.
>> >>>>>
>> >>>>> -Wei
>> >>>>>
>> >>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>> >>>>>
>> >>>>>> Hi,
>> >>>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>>> If nobody object, I will merge into master in 48 hours.
>> >>>>>> The purpose is :
>> >>>>>>
>> >>>>>> Virtual machines are running on the same storage device (local
>> storage
>> >>>> or
>> >>>>>> share strage). Because of the rate limitation of device (such as
>> >> iops),
>> >>>> if
>> >>>>>> one VM has large disk operation, it may affect the disk
>> performance of
>> >>>>>> other VMs running on the same storage device.
>> >>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>> VMs.
>> >>>>>>
>> >>>>>> The feature includes:
>> >>>>>>
>> >>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>> >>>>>> configuration)
>> >>>>>> (2) change the maximum rate of VMs
>> >>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>> >>>>>> FS (I will update later) :
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>> Merge check list :-
>> >>>>>>
>> >>>>>> * Did you check the branch's RAT execution success?
>> >>>>>> Yes
>> >>>>>>
>> >>>>>> * Are there new dependencies introduced?
>> >>>>>> No
>> >>>>>>
>> >>>>>> * What automated testing (unit and integration) is included in the
>> new
>> >>>>>> feature?
>> >>>>>> Unit tests are added.
>> >>>>>>
>> >>>>>> * What testing has been done to check for potential regressions?
>> >>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>> (2) VM operations, including
>> >>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>> >>>>>> (3) Volume operations, including
>> >>>>>> Attach, Detach
>> >>>>>>
>> >>>>>> To review the code, you can try
>> >>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>> >>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>> >>>>>>
>> >>>>>> Best regards,
>> >>>>>> Wei
>> >>>>>>
>> >>>>>> [1]
>> >>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>> >(CLOUDSTACK-1301
>> >> -
>> >>>> VM Disk I/O Throttling)
>> >>>>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *™*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
What do you think, Wei?
Should we come up with a way for only one feature (yours or mine) to be
used at a time on the new Disk Offering dialog?
Since most storage-side provisioned IOPS don't break it down into separate
read and write categories, I think that's the way to go (only one feature
or the other).
Any suggestions from a usability standpoint how we want to implement this?
It could be as simple as a radio button to turn on your feature and mine
off or vice versa.
Thanks!
On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I agree -- I can't image a situation where you would want to use IOPS
> provisioned by both the hypervisor and storage. There are two points of
> concern -- the UI and the management server. We have to ensure that the
> user can't create a VM from a compute/disk offering combination where
> hypervisor throttled I/O would contradict/conflict with storage provisioned
> IOPS. I think this functional conflict must be resolved in the management
> server to ensure that API calls are properly validated with a UX that
> avoids user confusion. Have Wei and you worked out an approach to
> resolving this conflict?
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Wei has sent me the screen shots.
> >
> > I don't support Compute Offerings for 4.2, so that's not an issue here.
> >
> > I do support Disk Offerings.
> >
> > It looks like Wei has added four new fields to the Disk Offering.
> >
> > I have added three (Min, Max, and Burst IOPS).
> >
> > We just need to decide if we should toggle between his and mine.
> >
> > I doubt a user would want to use both features at the same time.
> >
> >
> > On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Have Wei and you figured out the system level as well (e.g. allowing
> >> either storage provisioned IOPS or hypervisor throttling, but no both)?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Perhaps Wei could send me some screen shots of what he's changed in the
> >> GUI
> >>> for his feature?
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Wei,
> >>>>
> >>>> Have Mike Tutkowski and you reconciled the potential conflict between
> a
> >>>> throttled I/O VM and a provisioned IOPs volume? If so, what solution
> >> did
> >>>> you select?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
> >>>>
> >>>>> Guys,
> >>>>>
> >>>>> I would like to merge disk_io_throttling branch into master.
> >>>>> Please review the code on https://reviews.apache.org/r/11782
> >>>>>
> >>>>> If nobody object, I will merge into master in 72 hours.
> >>>>>
> >>>>> -Wei
> >>>>>
> >>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >>>>>
> >>>>>> Hi,
> >>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>> The purpose is :
> >>>>>>
> >>>>>> Virtual machines are running on the same storage device (local
> storage
> >>>> or
> >>>>>> share strage). Because of the rate limitation of device (such as
> >> iops),
> >>>> if
> >>>>>> one VM has large disk operation, it may affect the disk performance
> of
> >>>>>> other VMs running on the same storage device.
> >>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
> VMs.
> >>>>>>
> >>>>>> The feature includes:
> >>>>>>
> >>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >>>>>> configuration)
> >>>>>> (2) change the maximum rate of VMs
> >>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>>>> FS (I will update later) :
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>> Merge check list :-
> >>>>>>
> >>>>>> * Did you check the branch's RAT execution success?
> >>>>>> Yes
> >>>>>>
> >>>>>> * Are there new dependencies introduced?
> >>>>>> No
> >>>>>>
> >>>>>> * What automated testing (unit and integration) is included in the
> new
> >>>>>> feature?
> >>>>>> Unit tests are added.
> >>>>>>
> >>>>>> * What testing has been done to check for potential regressions?
> >>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>> (2) VM operations, including
> >>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >>>>>> (3) Volume operations, including
> >>>>>> Attach, Detach
> >>>>>>
> >>>>>> To review the code, you can try
> >>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >>>>>>
> >>>>>> Best regards,
> >>>>>> Wei
> >>>>>>
> >>>>>> [1]
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>> [2] refs/heads/disk_io_throttling
> >>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
> >(CLOUDSTACK-1301
> >> -
> >>>> VM Disk I/O Throttling)
> >>>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *™*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Mike,
I agree -- I can't image a situation where you would want to use IOPS provisioned by both the hypervisor and storage. There are two points of concern -- the UI and the management server. We have to ensure that the user can't create a VM from a compute/disk offering combination where hypervisor throttled I/O would contradict/conflict with storage provisioned IOPS. I think this functional conflict must be resolved in the management server to ensure that API calls are properly validated with a UX that avoids user confusion. Have Wei and you worked out an approach to resolving this conflict?
Thanks,
-John
On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Wei has sent me the screen shots.
>
> I don't support Compute Offerings for 4.2, so that's not an issue here.
>
> I do support Disk Offerings.
>
> It looks like Wei has added four new fields to the Disk Offering.
>
> I have added three (Min, Max, and Burst IOPS).
>
> We just need to decide if we should toggle between his and mine.
>
> I doubt a user would want to use both features at the same time.
>
>
> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Have Wei and you figured out the system level as well (e.g. allowing
>> either storage provisioned IOPS or hypervisor throttling, but no both)?
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Perhaps Wei could send me some screen shots of what he's changed in the
>> GUI
>>> for his feature?
>>>
>>> Thanks!
>>>
>>>
>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Wei,
>>>>
>>>> Have Mike Tutkowski and you reconciled the potential conflict between a
>>>> throttled I/O VM and a provisioned IOPs volume? If so, what solution
>> did
>>>> you select?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>>
>>>>> Guys,
>>>>>
>>>>> I would like to merge disk_io_throttling branch into master.
>>>>> Please review the code on https://reviews.apache.org/r/11782
>>>>>
>>>>> If nobody object, I will merge into master in 72 hours.
>>>>>
>>>>> -Wei
>>>>>
>>>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>>>
>>>>>> Hi,
>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>> The purpose is :
>>>>>>
>>>>>> Virtual machines are running on the same storage device (local storage
>>>> or
>>>>>> share strage). Because of the rate limitation of device (such as
>> iops),
>>>> if
>>>>>> one VM has large disk operation, it may affect the disk performance of
>>>>>> other VMs running on the same storage device.
>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>>>
>>>>>> The feature includes:
>>>>>>
>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>> configuration)
>>>>>> (2) change the maximum rate of VMs
>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>> FS (I will update later) :
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>> Merge check list :-
>>>>>>
>>>>>> * Did you check the branch's RAT execution success?
>>>>>> Yes
>>>>>>
>>>>>> * Are there new dependencies introduced?
>>>>>> No
>>>>>>
>>>>>> * What automated testing (unit and integration) is included in the new
>>>>>> feature?
>>>>>> Unit tests are added.
>>>>>>
>>>>>> * What testing has been done to check for potential regressions?
>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>> (2) VM operations, including
>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>> (3) Volume operations, including
>>>>>> Attach, Detach
>>>>>>
>>>>>> To review the code, you can try
>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>
>>>>>> Best regards,
>>>>>> Wei
>>>>>>
>>>>>> [1]
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>> [2] refs/heads/disk_io_throttling
>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301
>> -
>>>> VM Disk I/O Throttling)
>>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Wei has sent me the screen shots.
I don't support Compute Offerings for 4.2, so that's not an issue here.
I do support Disk Offerings.
It looks like Wei has added four new fields to the Disk Offering.
I have added three (Min, Max, and Burst IOPS).
We just need to decide if we should toggle between his and mine.
I doubt a user would want to use both features at the same time.
On Mon, Jun 10, 2013 at 12:30 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Have Wei and you figured out the system level as well (e.g. allowing
> either storage provisioned IOPS or hypervisor throttling, but no both)?
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Perhaps Wei could send me some screen shots of what he's changed in the
> GUI
> > for his feature?
> >
> > Thanks!
> >
> >
> > On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Wei,
> >>
> >> Have Mike Tutkowski and you reconciled the potential conflict between a
> >> throttled I/O VM and a provisioned IOPs volume? If so, what solution
> did
> >> you select?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
> >>
> >>> Guys,
> >>>
> >>> I would like to merge disk_io_throttling branch into master.
> >>> Please review the code on https://reviews.apache.org/r/11782
> >>>
> >>> If nobody object, I will merge into master in 72 hours.
> >>>
> >>> -Wei
> >>>
> >>> 2013/5/30 Wei ZHOU <us...@gmail.com>
> >>>
> >>>> Hi,
> >>>> I would like to merge disk_io_throttling branch into master.
> >>>> If nobody object, I will merge into master in 48 hours.
> >>>> The purpose is :
> >>>>
> >>>> Virtual machines are running on the same storage device (local storage
> >> or
> >>>> share strage). Because of the rate limitation of device (such as
> iops),
> >> if
> >>>> one VM has large disk operation, it may affect the disk performance of
> >>>> other VMs running on the same storage device.
> >>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
> >>>>
> >>>> The feature includes:
> >>>>
> >>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >>>> configuration)
> >>>> (2) change the maximum rate of VMs
> >>>> (3) limit the disk rate (total bps and iops)
> >>>> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>> FS (I will update later) :
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>> Merge check list :-
> >>>>
> >>>> * Did you check the branch's RAT execution success?
> >>>> Yes
> >>>>
> >>>> * Are there new dependencies introduced?
> >>>> No
> >>>>
> >>>> * What automated testing (unit and integration) is included in the new
> >>>> feature?
> >>>> Unit tests are added.
> >>>>
> >>>> * What testing has been done to check for potential regressions?
> >>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>> (2) VM operations, including
> >>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >>>> (3) Volume operations, including
> >>>> Attach, Detach
> >>>>
> >>>> To review the code, you can try
> >>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >>>> f2e5591b710d04cc86815044f5823e73a4a58944
> >>>>
> >>>> Best regards,
> >>>> Wei
> >>>>
> >>>> [1]
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>> [2] refs/heads/disk_io_throttling
> >>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301
> -
> >> VM Disk I/O Throttling)
> >>>>
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Mike,
Have Wei and you figured out the system level as well (e.g. allowing either storage provisioned IOPS or hypervisor throttling, but no both)?
Thanks,
-John
On Jun 10, 2013, at 2:12 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Perhaps Wei could send me some screen shots of what he's changed in the GUI
> for his feature?
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com> wrote:
>
>> Wei,
>>
>> Have Mike Tutkowski and you reconciled the potential conflict between a
>> throttled I/O VM and a provisioned IOPs volume? If so, what solution did
>> you select?
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Guys,
>>>
>>> I would like to merge disk_io_throttling branch into master.
>>> Please review the code on https://reviews.apache.org/r/11782
>>>
>>> If nobody object, I will merge into master in 72 hours.
>>>
>>> -Wei
>>>
>>> 2013/5/30 Wei ZHOU <us...@gmail.com>
>>>
>>>> Hi,
>>>> I would like to merge disk_io_throttling branch into master.
>>>> If nobody object, I will merge into master in 48 hours.
>>>> The purpose is :
>>>>
>>>> Virtual machines are running on the same storage device (local storage
>> or
>>>> share strage). Because of the rate limitation of device (such as iops),
>> if
>>>> one VM has large disk operation, it may affect the disk performance of
>>>> other VMs running on the same storage device.
>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>
>>>> The feature includes:
>>>>
>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>> configuration)
>>>> (2) change the maximum rate of VMs
>>>> (3) limit the disk rate (total bps and iops)
>>>> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>> FS (I will update later) :
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> Merge check list :-
>>>>
>>>> * Did you check the branch's RAT execution success?
>>>> Yes
>>>>
>>>> * Are there new dependencies introduced?
>>>> No
>>>>
>>>> * What automated testing (unit and integration) is included in the new
>>>> feature?
>>>> Unit tests are added.
>>>>
>>>> * What testing has been done to check for potential regressions?
>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>> (2) VM operations, including
>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>> (3) Volume operations, including
>>>> Attach, Detach
>>>>
>>>> To review the code, you can try
>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>
>>>> Best regards,
>>>> Wei
>>>>
>>>> [1]
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> [2] refs/heads/disk_io_throttling
>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301 -
>> VM Disk I/O Throttling)
>>>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by Mike Tutkowski <mi...@solidfire.com>.
Perhaps Wei could send me some screen shots of what he's changed in the GUI
for his feature?
Thanks!
On Mon, Jun 10, 2013 at 11:56 AM, John Burwell <jb...@basho.com> wrote:
> Wei,
>
> Have Mike Tutkowski and you reconciled the potential conflict between a
> throttled I/O VM and a provisioned IOPs volume? If so, what solution did
> you select?
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
>
> > Guys,
> >
> > I would like to merge disk_io_throttling branch into master.
> > Please review the code on https://reviews.apache.org/r/11782
> >
> > If nobody object, I will merge into master in 72 hours.
> >
> > -Wei
> >
> > 2013/5/30 Wei ZHOU <us...@gmail.com>
> >
> >> Hi,
> >> I would like to merge disk_io_throttling branch into master.
> >> If nobody object, I will merge into master in 48 hours.
> >> The purpose is :
> >>
> >> Virtual machines are running on the same storage device (local storage
> or
> >> share strage). Because of the rate limitation of device (such as iops),
> if
> >> one VM has large disk operation, it may affect the disk performance of
> >> other VMs running on the same storage device.
> >> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
> >>
> >> The feature includes:
> >>
> >> (1) set the maximum rate of VMs (in disk_offering, and global
> >> configuration)
> >> (2) change the maximum rate of VMs
> >> (3) limit the disk rate (total bps and iops)
> >> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >> FS (I will update later) :
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >> Merge check list :-
> >>
> >> * Did you check the branch's RAT execution success?
> >> Yes
> >>
> >> * Are there new dependencies introduced?
> >> No
> >>
> >> * What automated testing (unit and integration) is included in the new
> >> feature?
> >> Unit tests are added.
> >>
> >> * What testing has been done to check for potential regressions?
> >> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >> (2) VM operations, including
> >> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >> (3) Volume operations, including
> >> Attach, Detach
> >>
> >> To review the code, you can try
> >> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> >> f2e5591b710d04cc86815044f5823e73a4a58944
> >>
> >> Best regards,
> >> Wei
> >>
> >> [1]
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >> [2] refs/heads/disk_io_throttling
> >> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301 -
> VM Disk I/O Throttling)
> >>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Posted by John Burwell <jb...@basho.com>.
Wei,
Have Mike Tutkowski and you reconciled the potential conflict between a throttled I/O VM and a provisioned IOPs volume? If so, what solution did you select?
Thanks,
-John
On Jun 10, 2013, at 1:54 PM, Wei ZHOU <us...@gmail.com> wrote:
> Guys,
>
> I would like to merge disk_io_throttling branch into master.
> Please review the code on https://reviews.apache.org/r/11782
>
> If nobody object, I will merge into master in 72 hours.
>
> -Wei
>
> 2013/5/30 Wei ZHOU <us...@gmail.com>
>
>> Hi,
>> I would like to merge disk_io_throttling branch into master.
>> If nobody object, I will merge into master in 48 hours.
>> The purpose is :
>>
>> Virtual machines are running on the same storage device (local storage or
>> share strage). Because of the rate limitation of device (such as iops), if
>> one VM has large disk operation, it may affect the disk performance of
>> other VMs running on the same storage device.
>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>
>> The feature includes:
>>
>> (1) set the maximum rate of VMs (in disk_offering, and global
>> configuration)
>> (2) change the maximum rate of VMs
>> (3) limit the disk rate (total bps and iops)
>> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>> FS (I will update later) :
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> Merge check list :-
>>
>> * Did you check the branch's RAT execution success?
>> Yes
>>
>> * Are there new dependencies introduced?
>> No
>>
>> * What automated testing (unit and integration) is included in the new
>> feature?
>> Unit tests are added.
>>
>> * What testing has been done to check for potential regressions?
>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> (2) VM operations, including
>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>> (3) Volume operations, including
>> Attach, Detach
>>
>> To review the code, you can try
>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>> f2e5591b710d04cc86815044f5823e73a4a58944
>>
>> Best regards,
>> Wei
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> [2] refs/heads/disk_io_throttling
>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301 - VM Disk I/O Throttling)
>>