Change Volume IOPS on fly without detaching the disk feature.

[Punith S <pu...@cloudbyte.com>]

hi guys,

we are having a fixed max iops for each volume being attached to the
instance in managed storage,
so this a problem where we are making users to pre allocate the iops of the
disk without having an option to change or resize it, similar to the size
metric.

so i would like to introduce a new feature which enables to change or
resize the volume iops on fly without detaching the datadisk of the VM with
zero downtime where performance of the datadisk can be altered at any point
with the available iops of the primary storage pool, which is similar in
resizing the volume or datadisk of the vm , where in latter we have to
detach the datadisk.

what do you guys think about this feature ? any feedback ?

thanks,

-- 
regards,

punith s
cloudbyte.com

Re: Change Volume IOPS on fly without detaching the disk feature.

[Wei ZHOU <us...@gmail.com>]

Brilliant!


2014-03-06 12:07 GMT+01:00 Wido den Hollander <wi...@widodh.nl>:

>
>
> On 03/06/2014 11:48 AM, Wei ZHOU wrote:
>
>> awesome!
>>
>> Can you implement cpu tuning or network QoS as well ? Thanks!
>>
>>
> Yes, I was planning on adding multiple methods at once with a couple of
> patches.
>
>
>  -Wei
>>
>>
>>

Re: Change Volume IOPS on fly without detaching the disk feature.

[Wido den Hollander <wi...@widodh.nl>]

On 03/06/2014 11:48 AM, Wei ZHOU wrote:
> awesome!
>
> Can you implement cpu tuning or network QoS as well ? Thanks!
>

Yes, I was planning on adding multiple methods at once with a couple of 
patches.

> -Wei
>
>
> 2014-03-06 11:42 GMT+01:00 Wido den Hollander <wi...@widodh.nl>:
>
>>
>>
>> On 03/05/2014 07:18 PM, Marcus wrote:
>>
>>> For the hypervisor version of throttling, we just need
>>> ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
>>> volume uuid/path, so that when we change offerings on the agent side
>>> we have the info we need to update libvirt with the new iops/bytes
>>> settings. We also need the libvirt java bindings to do so, per
>>> previous discussion.
>>>
>>>
>> I'm already working on the patch: https://github.com/wido/
>> libvirt-java/tree/change-iops
>>
>> It's not so hard to implement it seems. Hopefully I'll have it ready after
>> the weekend.
>>
>>
>>   On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
>>>
>>>> Wouldn't this be implemented as just changing disk offerings? The
>>>> resizeVolume API call already allows you to switch disk offerings, we
>>>> just need to add a hook in there to optionally call the storage driver
>>>> (If volume is deployed to a primary storage) to make an update to the
>>>> iops properties on the backend storage. Come to think of it, depending
>>>> on how storage drivers are implementing the iops/limits feature,
>>>> resizeVolume might be breaking this, or simply requiring a reboot to
>>>> apply. That is, if the storage driver is setting the iops just once
>>>> upon volume creation, it's probably breaking when a user moves a disk
>>>> between offerings that may have alternate iops limits (this is
>>>> probably not the case for hypervisor throttling, as that's applied
>>>> from whatever is current when the VM starts up).
>>>>
>>>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
>>>> <mi...@solidfire.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Perhaps I'm not following this correctly, but I'm a bit lost on why we
>>>>> are
>>>>> talking about changing settings on running VMs.
>>>>>
>>>>>   From what I understand, you are a representative of a storage vendor
>>>>> that
>>>>> has a rate-limiting feature. You want to be able to not only set the Max
>>>>> IOPS, but also adjust them. Is this true?
>>>>>
>>>>> If so, I totally agree. SolidFire has control over Min and Max IOPS and
>>>>> it
>>>>> is on my to-do list to add support into CloudStack to be able to
>>>>> dynamically change these values (right now customers do this from the
>>>>> SolidFire API or its GUI).
>>>>>
>>>>> If you would like to work on this feature, that would be great. I'd be
>>>>> happy to review your design and code.
>>>>>
>>>>> One complication is that we are looking at adding support for generic
>>>>> key/value pairs for storage plug-ins in 4.5 and this would effectively
>>>>> remove the need to have Min and Max IOPS as "special" fields in the
>>>>> CloudStack API and GUI.
>>>>>
>>>>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
>>>>> discussed this generic-properties concept. It would be good to get his
>>>>> feedback on how we might go about dynamically updating storage-plug-in
>>>>> key/value pairs.
>>>>>
>>>>> Thanks!
>>>>> Mike
>>>>>
>>>>>
>>>>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
>>>>> wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>>>>>>
>>>>>>   I was thinking about it last week.
>>>>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
>>>>>>> vms,
>>>>>>> but virsh command line and libvirt API supports it.
>>>>>>> so the sulution are
>>>>>>> (1) change libvirt-java to support it, and make it released in the
>>>>>>> next
>>>>>>> version. Maybe Wido can help us.
>>>>>>>
>>>>>>>
>>>>>> Sure! That seems the best way forward. What is currently lacking in the
>>>>>> libvirt-java bindings?
>>>>>>
>>>>>>
>>>>>>    (2) call virsh command line.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>   Please, please, do not do that. That's very hacky. We should really
>>>>>> keep
>>>>>> using the libvirt-java bindings and stay away from invoking binaries.
>>>>>>
>>>>>> Wido
>>>>>>
>>>>>>
>>>>>>    -Wei
>>>>>>
>>>>>>>
>>>>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>>>>>>
>>>>>>>    hi guys,
>>>>>>>
>>>>>>>>
>>>>>>>> we are having a fixed max iops for each volume being attached to the
>>>>>>>> instance in managed storage,
>>>>>>>> so this a problem where we are making users to pre allocate the iops
>>>>>>>> of
>>>>>>>> the
>>>>>>>> disk without having an option to change or resize it, similar to the
>>>>>>>> size
>>>>>>>> metric.
>>>>>>>>
>>>>>>>> so i would like to introduce a new feature which enables to change or
>>>>>>>> resize the volume iops on fly without detaching the datadisk of the
>>>>>>>> VM
>>>>>>>> with
>>>>>>>> zero downtime where performance of the datadisk can be altered at any
>>>>>>>> point
>>>>>>>> with the available iops of the primary storage pool, which is
>>>>>>>> similar in
>>>>>>>> resizing the volume or datadisk of the vm , where in latter we have
>>>>>>>> to
>>>>>>>> detach the datadisk.
>>>>>>>>
>>>>>>>> what do you guys think about this feature ? any feedback ?
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> --
>>>>>>>> regards,
>>>>>>>>
>>>>>>>> punith s
>>>>>>>> cloudbyte.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *(tm)*
>>>>>
>>>>
>

Re: Change Volume IOPS on fly without detaching the disk feature.

[Wei ZHOU <us...@gmail.com>]

awesome!

Can you implement cpu tuning or network QoS as well ? Thanks!

-Wei


2014-03-06 11:42 GMT+01:00 Wido den Hollander <wi...@widodh.nl>:

>
>
> On 03/05/2014 07:18 PM, Marcus wrote:
>
>> For the hypervisor version of throttling, we just need
>> ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
>> volume uuid/path, so that when we change offerings on the agent side
>> we have the info we need to update libvirt with the new iops/bytes
>> settings. We also need the libvirt java bindings to do so, per
>> previous discussion.
>>
>>
> I'm already working on the patch: https://github.com/wido/
> libvirt-java/tree/change-iops
>
> It's not so hard to implement it seems. Hopefully I'll have it ready after
> the weekend.
>
>
>  On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
>>
>>> Wouldn't this be implemented as just changing disk offerings? The
>>> resizeVolume API call already allows you to switch disk offerings, we
>>> just need to add a hook in there to optionally call the storage driver
>>> (If volume is deployed to a primary storage) to make an update to the
>>> iops properties on the backend storage. Come to think of it, depending
>>> on how storage drivers are implementing the iops/limits feature,
>>> resizeVolume might be breaking this, or simply requiring a reboot to
>>> apply. That is, if the storage driver is setting the iops just once
>>> upon volume creation, it's probably breaking when a user moves a disk
>>> between offerings that may have alternate iops limits (this is
>>> probably not the case for hypervisor throttling, as that's applied
>>> from whatever is current when the VM starts up).
>>>
>>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
>>> <mi...@solidfire.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Perhaps I'm not following this correctly, but I'm a bit lost on why we
>>>> are
>>>> talking about changing settings on running VMs.
>>>>
>>>>  From what I understand, you are a representative of a storage vendor
>>>> that
>>>> has a rate-limiting feature. You want to be able to not only set the Max
>>>> IOPS, but also adjust them. Is this true?
>>>>
>>>> If so, I totally agree. SolidFire has control over Min and Max IOPS and
>>>> it
>>>> is on my to-do list to add support into CloudStack to be able to
>>>> dynamically change these values (right now customers do this from the
>>>> SolidFire API or its GUI).
>>>>
>>>> If you would like to work on this feature, that would be great. I'd be
>>>> happy to review your design and code.
>>>>
>>>> One complication is that we are looking at adding support for generic
>>>> key/value pairs for storage plug-ins in 4.5 and this would effectively
>>>> remove the need to have Min and Max IOPS as "special" fields in the
>>>> CloudStack API and GUI.
>>>>
>>>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
>>>> discussed this generic-properties concept. It would be good to get his
>>>> feedback on how we might go about dynamically updating storage-plug-in
>>>> key/value pairs.
>>>>
>>>> Thanks!
>>>> Mike
>>>>
>>>>
>>>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>>>>>
>>>>>  I was thinking about it last week.
>>>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
>>>>>> vms,
>>>>>> but virsh command line and libvirt API supports it.
>>>>>> so the sulution are
>>>>>> (1) change libvirt-java to support it, and make it released in the
>>>>>> next
>>>>>> version. Maybe Wido can help us.
>>>>>>
>>>>>>
>>>>> Sure! That seems the best way forward. What is currently lacking in the
>>>>> libvirt-java bindings?
>>>>>
>>>>>
>>>>>   (2) call virsh command line.
>>>>>
>>>>>>
>>>>>>
>>>>>>  Please, please, do not do that. That's very hacky. We should really
>>>>> keep
>>>>> using the libvirt-java bindings and stay away from invoking binaries.
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>>   -Wei
>>>>>
>>>>>>
>>>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>>>>>
>>>>>>   hi guys,
>>>>>>
>>>>>>>
>>>>>>> we are having a fixed max iops for each volume being attached to the
>>>>>>> instance in managed storage,
>>>>>>> so this a problem where we are making users to pre allocate the iops
>>>>>>> of
>>>>>>> the
>>>>>>> disk without having an option to change or resize it, similar to the
>>>>>>> size
>>>>>>> metric.
>>>>>>>
>>>>>>> so i would like to introduce a new feature which enables to change or
>>>>>>> resize the volume iops on fly without detaching the datadisk of the
>>>>>>> VM
>>>>>>> with
>>>>>>> zero downtime where performance of the datadisk can be altered at any
>>>>>>> point
>>>>>>> with the available iops of the primary storage pool, which is
>>>>>>> similar in
>>>>>>> resizing the volume or datadisk of the vm , where in latter we have
>>>>>>> to
>>>>>>> detach the datadisk.
>>>>>>>
>>>>>>> what do you guys think about this feature ? any feedback ?
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> --
>>>>>>> regards,
>>>>>>>
>>>>>>> punith s
>>>>>>> cloudbyte.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*
>>>>
>>>

Re: Change Volume IOPS on fly without detaching the disk feature.

[Mike Tutkowski <mi...@solidfire.com>]

Sounds good - let's plan on that for 4.5. :)


On Thu, Mar 6, 2014 at 10:15 PM, Punith S <pu...@cloudbyte.com> wrote:

> yea sure , will target this for 4.5.
>
> thanks
>
>
> On Thu, Mar 6, 2014 at 8:37 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> > wrote:
>
> > Just to clarify, Punith: You are correct that changing storage QoS
> > dynamically will be a more time-consuming task than adding that kind of
> > support on the hypervisor side. That's why I say with the Feature Freeze
> > date as it is for 4.4 that we should look to address this in 4.5.
> >
> > Thanks
> >
> >
> > On Thu, Mar 6, 2014 at 3:42 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >
> > >
> > >
> > > On 03/05/2014 07:18 PM, Marcus wrote:
> > >
> > >> For the hypervisor version of throttling, we just need
> > >> ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
> > >> volume uuid/path, so that when we change offerings on the agent side
> > >> we have the info we need to update libvirt with the new iops/bytes
> > >> settings. We also need the libvirt java bindings to do so, per
> > >> previous discussion.
> > >>
> > >>
> > > I'm already working on the patch: https://github.com/wido/
> > > libvirt-java/tree/change-iops
> > >
> > > It's not so hard to implement it seems. Hopefully I'll have it ready
> > after
> > > the weekend.
> > >
> > >
> > >  On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
> > >>
> > >>> Wouldn't this be implemented as just changing disk offerings? The
> > >>> resizeVolume API call already allows you to switch disk offerings, we
> > >>> just need to add a hook in there to optionally call the storage
> driver
> > >>> (If volume is deployed to a primary storage) to make an update to the
> > >>> iops properties on the backend storage. Come to think of it,
> depending
> > >>> on how storage drivers are implementing the iops/limits feature,
> > >>> resizeVolume might be breaking this, or simply requiring a reboot to
> > >>> apply. That is, if the storage driver is setting the iops just once
> > >>> upon volume creation, it's probably breaking when a user moves a disk
> > >>> between offerings that may have alternate iops limits (this is
> > >>> probably not the case for hypervisor throttling, as that's applied
> > >>> from whatever is current when the VM starts up).
> > >>>
> > >>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
> > >>> <mi...@solidfire.com> wrote:
> > >>>
> > >>>> Hi,
> > >>>>
> > >>>> Perhaps I'm not following this correctly, but I'm a bit lost on why
> we
> > >>>> are
> > >>>> talking about changing settings on running VMs.
> > >>>>
> > >>>>  From what I understand, you are a representative of a storage
> vendor
> > >>>> that
> > >>>> has a rate-limiting feature. You want to be able to not only set the
> > Max
> > >>>> IOPS, but also adjust them. Is this true?
> > >>>>
> > >>>> If so, I totally agree. SolidFire has control over Min and Max IOPS
> > and
> > >>>> it
> > >>>> is on my to-do list to add support into CloudStack to be able to
> > >>>> dynamically change these values (right now customers do this from
> the
> > >>>> SolidFire API or its GUI).
> > >>>>
> > >>>> If you would like to work on this feature, that would be great. I'd
> be
> > >>>> happy to review your design and code.
> > >>>>
> > >>>> One complication is that we are looking at adding support for
> generic
> > >>>> key/value pairs for storage plug-ins in 4.5 and this would
> effectively
> > >>>> remove the need to have Min and Max IOPS as "special" fields in the
> > >>>> CloudStack API and GUI.
> > >>>>
> > >>>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
> > >>>> discussed this generic-properties concept. It would be good to get
> his
> > >>>> feedback on how we might go about dynamically updating
> storage-plug-in
> > >>>> key/value pairs.
> > >>>>
> > >>>> Thanks!
> > >>>> Mike
> > >>>>
> > >>>>
> > >>>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>>>
> > >>>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
> > >>>>>
> > >>>>>  I was thinking about it last week.
> > >>>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on
> running
> > >>>>>> vms,
> > >>>>>> but virsh command line and libvirt API supports it.
> > >>>>>> so the sulution are
> > >>>>>> (1) change libvirt-java to support it, and make it released in the
> > >>>>>> next
> > >>>>>> version. Maybe Wido can help us.
> > >>>>>>
> > >>>>>>
> > >>>>> Sure! That seems the best way forward. What is currently lacking in
> > the
> > >>>>> libvirt-java bindings?
> > >>>>>
> > >>>>>
> > >>>>>   (2) call virsh command line.
> > >>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>  Please, please, do not do that. That's very hacky. We should
> really
> > >>>>> keep
> > >>>>> using the libvirt-java bindings and stay away from invoking
> binaries.
> > >>>>>
> > >>>>> Wido
> > >>>>>
> > >>>>>
> > >>>>>   -Wei
> > >>>>>
> > >>>>>>
> > >>>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
> > >>>>>>
> > >>>>>>   hi guys,
> > >>>>>>
> > >>>>>>>
> > >>>>>>> we are having a fixed max iops for each volume being attached to
> > the
> > >>>>>>> instance in managed storage,
> > >>>>>>> so this a problem where we are making users to pre allocate the
> > iops
> > >>>>>>> of
> > >>>>>>> the
> > >>>>>>> disk without having an option to change or resize it, similar to
> > the
> > >>>>>>> size
> > >>>>>>> metric.
> > >>>>>>>
> > >>>>>>> so i would like to introduce a new feature which enables to
> change
> > or
> > >>>>>>> resize the volume iops on fly without detaching the datadisk of
> the
> > >>>>>>> VM
> > >>>>>>> with
> > >>>>>>> zero downtime where performance of the datadisk can be altered at
> > any
> > >>>>>>> point
> > >>>>>>> with the available iops of the primary storage pool, which is
> > >>>>>>> similar in
> > >>>>>>> resizing the volume or datadisk of the vm , where in latter we
> have
> > >>>>>>> to
> > >>>>>>> detach the datadisk.
> > >>>>>>>
> > >>>>>>> what do you guys think about this feature ? any feedback ?
> > >>>>>>>
> > >>>>>>> thanks,
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> regards,
> > >>>>>>>
> > >>>>>>> punith s
> > >>>>>>> cloudbyte.com
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>> --
> > >>>> *Mike Tutkowski*
> > >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>> e: mike.tutkowski@solidfire.com
> > >>>> o: 303.746.7302
> > >>>> Advancing the way the world uses the
> > >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>> *(tm)*
> > >>>>
> > >>>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
> >
>
>
>
> --
> regards,
>
> punith s
> cloudbyte.com
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Punith S <pu...@cloudbyte.com>]

yea sure , will target this for 4.5.

thanks


On Thu, Mar 6, 2014 at 8:37 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:

> Just to clarify, Punith: You are correct that changing storage QoS
> dynamically will be a more time-consuming task than adding that kind of
> support on the hypervisor side. That's why I say with the Feature Freeze
> date as it is for 4.4 that we should look to address this in 4.5.
>
> Thanks
>
>
> On Thu, Mar 6, 2014 at 3:42 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
> >
> >
> > On 03/05/2014 07:18 PM, Marcus wrote:
> >
> >> For the hypervisor version of throttling, we just need
> >> ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
> >> volume uuid/path, so that when we change offerings on the agent side
> >> we have the info we need to update libvirt with the new iops/bytes
> >> settings. We also need the libvirt java bindings to do so, per
> >> previous discussion.
> >>
> >>
> > I'm already working on the patch: https://github.com/wido/
> > libvirt-java/tree/change-iops
> >
> > It's not so hard to implement it seems. Hopefully I'll have it ready
> after
> > the weekend.
> >
> >
> >  On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
> >>
> >>> Wouldn't this be implemented as just changing disk offerings? The
> >>> resizeVolume API call already allows you to switch disk offerings, we
> >>> just need to add a hook in there to optionally call the storage driver
> >>> (If volume is deployed to a primary storage) to make an update to the
> >>> iops properties on the backend storage. Come to think of it, depending
> >>> on how storage drivers are implementing the iops/limits feature,
> >>> resizeVolume might be breaking this, or simply requiring a reboot to
> >>> apply. That is, if the storage driver is setting the iops just once
> >>> upon volume creation, it's probably breaking when a user moves a disk
> >>> between offerings that may have alternate iops limits (this is
> >>> probably not the case for hypervisor throttling, as that's applied
> >>> from whatever is current when the VM starts up).
> >>>
> >>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
> >>> <mi...@solidfire.com> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> Perhaps I'm not following this correctly, but I'm a bit lost on why we
> >>>> are
> >>>> talking about changing settings on running VMs.
> >>>>
> >>>>  From what I understand, you are a representative of a storage vendor
> >>>> that
> >>>> has a rate-limiting feature. You want to be able to not only set the
> Max
> >>>> IOPS, but also adjust them. Is this true?
> >>>>
> >>>> If so, I totally agree. SolidFire has control over Min and Max IOPS
> and
> >>>> it
> >>>> is on my to-do list to add support into CloudStack to be able to
> >>>> dynamically change these values (right now customers do this from the
> >>>> SolidFire API or its GUI).
> >>>>
> >>>> If you would like to work on this feature, that would be great. I'd be
> >>>> happy to review your design and code.
> >>>>
> >>>> One complication is that we are looking at adding support for generic
> >>>> key/value pairs for storage plug-ins in 4.5 and this would effectively
> >>>> remove the need to have Min and Max IOPS as "special" fields in the
> >>>> CloudStack API and GUI.
> >>>>
> >>>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
> >>>> discussed this generic-properties concept. It would be good to get his
> >>>> feedback on how we might go about dynamically updating storage-plug-in
> >>>> key/value pairs.
> >>>>
> >>>> Thanks!
> >>>> Mike
> >>>>
> >>>>
> >>>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
> >>>> wrote:
> >>>>
> >>>>
> >>>>>
> >>>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
> >>>>>
> >>>>>  I was thinking about it last week.
> >>>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
> >>>>>> vms,
> >>>>>> but virsh command line and libvirt API supports it.
> >>>>>> so the sulution are
> >>>>>> (1) change libvirt-java to support it, and make it released in the
> >>>>>> next
> >>>>>> version. Maybe Wido can help us.
> >>>>>>
> >>>>>>
> >>>>> Sure! That seems the best way forward. What is currently lacking in
> the
> >>>>> libvirt-java bindings?
> >>>>>
> >>>>>
> >>>>>   (2) call virsh command line.
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>>  Please, please, do not do that. That's very hacky. We should really
> >>>>> keep
> >>>>> using the libvirt-java bindings and stay away from invoking binaries.
> >>>>>
> >>>>> Wido
> >>>>>
> >>>>>
> >>>>>   -Wei
> >>>>>
> >>>>>>
> >>>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
> >>>>>>
> >>>>>>   hi guys,
> >>>>>>
> >>>>>>>
> >>>>>>> we are having a fixed max iops for each volume being attached to
> the
> >>>>>>> instance in managed storage,
> >>>>>>> so this a problem where we are making users to pre allocate the
> iops
> >>>>>>> of
> >>>>>>> the
> >>>>>>> disk without having an option to change or resize it, similar to
> the
> >>>>>>> size
> >>>>>>> metric.
> >>>>>>>
> >>>>>>> so i would like to introduce a new feature which enables to change
> or
> >>>>>>> resize the volume iops on fly without detaching the datadisk of the
> >>>>>>> VM
> >>>>>>> with
> >>>>>>> zero downtime where performance of the datadisk can be altered at
> any
> >>>>>>> point
> >>>>>>> with the available iops of the primary storage pool, which is
> >>>>>>> similar in
> >>>>>>> resizing the volume or datadisk of the vm , where in latter we have
> >>>>>>> to
> >>>>>>> detach the datadisk.
> >>>>>>>
> >>>>>>> what do you guys think about this feature ? any feedback ?
> >>>>>>>
> >>>>>>> thanks,
> >>>>>>>
> >>>>>>> --
> >>>>>>> regards,
> >>>>>>>
> >>>>>>> punith s
> >>>>>>> cloudbyte.com
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*
> >>>>
> >>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
>



-- 
regards,

punith s
cloudbyte.com

Re: Change Volume IOPS on fly without detaching the disk feature.

[Mike Tutkowski <mi...@solidfire.com>]

Just to clarify, Punith: You are correct that changing storage QoS
dynamically will be a more time-consuming task than adding that kind of
support on the hypervisor side. That's why I say with the Feature Freeze
date as it is for 4.4 that we should look to address this in 4.5.

Thanks


On Thu, Mar 6, 2014 at 3:42 AM, Wido den Hollander <wi...@widodh.nl> wrote:

>
>
> On 03/05/2014 07:18 PM, Marcus wrote:
>
>> For the hypervisor version of throttling, we just need
>> ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
>> volume uuid/path, so that when we change offerings on the agent side
>> we have the info we need to update libvirt with the new iops/bytes
>> settings. We also need the libvirt java bindings to do so, per
>> previous discussion.
>>
>>
> I'm already working on the patch: https://github.com/wido/
> libvirt-java/tree/change-iops
>
> It's not so hard to implement it seems. Hopefully I'll have it ready after
> the weekend.
>
>
>  On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
>>
>>> Wouldn't this be implemented as just changing disk offerings? The
>>> resizeVolume API call already allows you to switch disk offerings, we
>>> just need to add a hook in there to optionally call the storage driver
>>> (If volume is deployed to a primary storage) to make an update to the
>>> iops properties on the backend storage. Come to think of it, depending
>>> on how storage drivers are implementing the iops/limits feature,
>>> resizeVolume might be breaking this, or simply requiring a reboot to
>>> apply. That is, if the storage driver is setting the iops just once
>>> upon volume creation, it's probably breaking when a user moves a disk
>>> between offerings that may have alternate iops limits (this is
>>> probably not the case for hypervisor throttling, as that's applied
>>> from whatever is current when the VM starts up).
>>>
>>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
>>> <mi...@solidfire.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Perhaps I'm not following this correctly, but I'm a bit lost on why we
>>>> are
>>>> talking about changing settings on running VMs.
>>>>
>>>>  From what I understand, you are a representative of a storage vendor
>>>> that
>>>> has a rate-limiting feature. You want to be able to not only set the Max
>>>> IOPS, but also adjust them. Is this true?
>>>>
>>>> If so, I totally agree. SolidFire has control over Min and Max IOPS and
>>>> it
>>>> is on my to-do list to add support into CloudStack to be able to
>>>> dynamically change these values (right now customers do this from the
>>>> SolidFire API or its GUI).
>>>>
>>>> If you would like to work on this feature, that would be great. I'd be
>>>> happy to review your design and code.
>>>>
>>>> One complication is that we are looking at adding support for generic
>>>> key/value pairs for storage plug-ins in 4.5 and this would effectively
>>>> remove the need to have Min and Max IOPS as "special" fields in the
>>>> CloudStack API and GUI.
>>>>
>>>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
>>>> discussed this generic-properties concept. It would be good to get his
>>>> feedback on how we might go about dynamically updating storage-plug-in
>>>> key/value pairs.
>>>>
>>>> Thanks!
>>>> Mike
>>>>
>>>>
>>>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>>>>>
>>>>>  I was thinking about it last week.
>>>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
>>>>>> vms,
>>>>>> but virsh command line and libvirt API supports it.
>>>>>> so the sulution are
>>>>>> (1) change libvirt-java to support it, and make it released in the
>>>>>> next
>>>>>> version. Maybe Wido can help us.
>>>>>>
>>>>>>
>>>>> Sure! That seems the best way forward. What is currently lacking in the
>>>>> libvirt-java bindings?
>>>>>
>>>>>
>>>>>   (2) call virsh command line.
>>>>>
>>>>>>
>>>>>>
>>>>>>  Please, please, do not do that. That's very hacky. We should really
>>>>> keep
>>>>> using the libvirt-java bindings and stay away from invoking binaries.
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>>   -Wei
>>>>>
>>>>>>
>>>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>>>>>
>>>>>>   hi guys,
>>>>>>
>>>>>>>
>>>>>>> we are having a fixed max iops for each volume being attached to the
>>>>>>> instance in managed storage,
>>>>>>> so this a problem where we are making users to pre allocate the iops
>>>>>>> of
>>>>>>> the
>>>>>>> disk without having an option to change or resize it, similar to the
>>>>>>> size
>>>>>>> metric.
>>>>>>>
>>>>>>> so i would like to introduce a new feature which enables to change or
>>>>>>> resize the volume iops on fly without detaching the datadisk of the
>>>>>>> VM
>>>>>>> with
>>>>>>> zero downtime where performance of the datadisk can be altered at any
>>>>>>> point
>>>>>>> with the available iops of the primary storage pool, which is
>>>>>>> similar in
>>>>>>> resizing the volume or datadisk of the vm , where in latter we have
>>>>>>> to
>>>>>>> detach the datadisk.
>>>>>>>
>>>>>>> what do you guys think about this feature ? any feedback ?
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> --
>>>>>>> regards,
>>>>>>>
>>>>>>> punith s
>>>>>>> cloudbyte.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*
>>>>
>>>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Wido den Hollander <wi...@widodh.nl>]

On 03/05/2014 07:18 PM, Marcus wrote:
> For the hypervisor version of throttling, we just need
> ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
> volume uuid/path, so that when we change offerings on the agent side
> we have the info we need to update libvirt with the new iops/bytes
> settings. We also need the libvirt java bindings to do so, per
> previous discussion.
>

I'm already working on the patch: 
https://github.com/wido/libvirt-java/tree/change-iops

It's not so hard to implement it seems. Hopefully I'll have it ready 
after the weekend.

> On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
>> Wouldn't this be implemented as just changing disk offerings? The
>> resizeVolume API call already allows you to switch disk offerings, we
>> just need to add a hook in there to optionally call the storage driver
>> (If volume is deployed to a primary storage) to make an update to the
>> iops properties on the backend storage. Come to think of it, depending
>> on how storage drivers are implementing the iops/limits feature,
>> resizeVolume might be breaking this, or simply requiring a reboot to
>> apply. That is, if the storage driver is setting the iops just once
>> upon volume creation, it's probably breaking when a user moves a disk
>> between offerings that may have alternate iops limits (this is
>> probably not the case for hypervisor throttling, as that's applied
>> from whatever is current when the VM starts up).
>>
>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>>> Hi,
>>>
>>> Perhaps I'm not following this correctly, but I'm a bit lost on why we are
>>> talking about changing settings on running VMs.
>>>
>>>  From what I understand, you are a representative of a storage vendor that
>>> has a rate-limiting feature. You want to be able to not only set the Max
>>> IOPS, but also adjust them. Is this true?
>>>
>>> If so, I totally agree. SolidFire has control over Min and Max IOPS and it
>>> is on my to-do list to add support into CloudStack to be able to
>>> dynamically change these values (right now customers do this from the
>>> SolidFire API or its GUI).
>>>
>>> If you would like to work on this feature, that would be great. I'd be
>>> happy to review your design and code.
>>>
>>> One complication is that we are looking at adding support for generic
>>> key/value pairs for storage plug-ins in 4.5 and this would effectively
>>> remove the need to have Min and Max IOPS as "special" fields in the
>>> CloudStack API and GUI.
>>>
>>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
>>> discussed this generic-properties concept. It would be good to get his
>>> feedback on how we might go about dynamically updating storage-plug-in
>>> key/value pairs.
>>>
>>> Thanks!
>>> Mike
>>>
>>>
>>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>>
>>>>
>>>>
>>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>>>>
>>>>> I was thinking about it last week.
>>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running vms,
>>>>> but virsh command line and libvirt API supports it.
>>>>> so the sulution are
>>>>> (1) change libvirt-java to support it, and make it released in the next
>>>>> version. Maybe Wido can help us.
>>>>>
>>>>
>>>> Sure! That seems the best way forward. What is currently lacking in the
>>>> libvirt-java bindings?
>>>>
>>>>
>>>>   (2) call virsh command line.
>>>>>
>>>>>
>>>> Please, please, do not do that. That's very hacky. We should really keep
>>>> using the libvirt-java bindings and stay away from invoking binaries.
>>>>
>>>> Wido
>>>>
>>>>
>>>>   -Wei
>>>>>
>>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>>>>
>>>>>   hi guys,
>>>>>>
>>>>>> we are having a fixed max iops for each volume being attached to the
>>>>>> instance in managed storage,
>>>>>> so this a problem where we are making users to pre allocate the iops of
>>>>>> the
>>>>>> disk without having an option to change or resize it, similar to the size
>>>>>> metric.
>>>>>>
>>>>>> so i would like to introduce a new feature which enables to change or
>>>>>> resize the volume iops on fly without detaching the datadisk of the VM
>>>>>> with
>>>>>> zero downtime where performance of the datadisk can be altered at any
>>>>>> point
>>>>>> with the available iops of the primary storage pool, which is similar in
>>>>>> resizing the volume or datadisk of the vm , where in latter we have to
>>>>>> detach the datadisk.
>>>>>>
>>>>>> what do you guys think about this feature ? any feedback ?
>>>>>>
>>>>>> thanks,
>>>>>>
>>>>>> --
>>>>>> regards,
>>>>>>
>>>>>> punith s
>>>>>> cloudbyte.com
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Marcus <sh...@gmail.com>]

For the hypervisor version of throttling, we just need
ResizeVolumeCommand to pass the VolumeObjectTO rather than just the
volume uuid/path, so that when we change offerings on the agent side
we have the info we need to update libvirt with the new iops/bytes
settings. We also need the libvirt java bindings to do so, per
previous discussion.

On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
> Wouldn't this be implemented as just changing disk offerings? The
> resizeVolume API call already allows you to switch disk offerings, we
> just need to add a hook in there to optionally call the storage driver
> (If volume is deployed to a primary storage) to make an update to the
> iops properties on the backend storage. Come to think of it, depending
> on how storage drivers are implementing the iops/limits feature,
> resizeVolume might be breaking this, or simply requiring a reboot to
> apply. That is, if the storage driver is setting the iops just once
> upon volume creation, it's probably breaking when a user moves a disk
> between offerings that may have alternate iops limits (this is
> probably not the case for hypervisor throttling, as that's applied
> from whatever is current when the VM starts up).
>
> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
>> Hi,
>>
>> Perhaps I'm not following this correctly, but I'm a bit lost on why we are
>> talking about changing settings on running VMs.
>>
>> From what I understand, you are a representative of a storage vendor that
>> has a rate-limiting feature. You want to be able to not only set the Max
>> IOPS, but also adjust them. Is this true?
>>
>> If so, I totally agree. SolidFire has control over Min and Max IOPS and it
>> is on my to-do list to add support into CloudStack to be able to
>> dynamically change these values (right now customers do this from the
>> SolidFire API or its GUI).
>>
>> If you would like to work on this feature, that would be great. I'd be
>> happy to review your design and code.
>>
>> One complication is that we are looking at adding support for generic
>> key/value pairs for storage plug-ins in 4.5 and this would effectively
>> remove the need to have Min and Max IOPS as "special" fields in the
>> CloudStack API and GUI.
>>
>> I'm going to CC Chris Suichll (from NetApp) as he and I have already
>> discussed this generic-properties concept. It would be good to get his
>> feedback on how we might go about dynamically updating storage-plug-in
>> key/value pairs.
>>
>> Thanks!
>> Mike
>>
>>
>> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>
>>>
>>>
>>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>>>
>>>> I was thinking about it last week.
>>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running vms,
>>>> but virsh command line and libvirt API supports it.
>>>> so the sulution are
>>>> (1) change libvirt-java to support it, and make it released in the next
>>>> version. Maybe Wido can help us.
>>>>
>>>
>>> Sure! That seems the best way forward. What is currently lacking in the
>>> libvirt-java bindings?
>>>
>>>
>>>  (2) call virsh command line.
>>>>
>>>>
>>> Please, please, do not do that. That's very hacky. We should really keep
>>> using the libvirt-java bindings and stay away from invoking binaries.
>>>
>>> Wido
>>>
>>>
>>>  -Wei
>>>>
>>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>>>
>>>>  hi guys,
>>>>>
>>>>> we are having a fixed max iops for each volume being attached to the
>>>>> instance in managed storage,
>>>>> so this a problem where we are making users to pre allocate the iops of
>>>>> the
>>>>> disk without having an option to change or resize it, similar to the size
>>>>> metric.
>>>>>
>>>>> so i would like to introduce a new feature which enables to change or
>>>>> resize the volume iops on fly without detaching the datadisk of the VM
>>>>> with
>>>>> zero downtime where performance of the datadisk can be altered at any
>>>>> point
>>>>> with the available iops of the primary storage pool, which is similar in
>>>>> resizing the volume or datadisk of the vm , where in latter we have to
>>>>> detach the datadisk.
>>>>>
>>>>> what do you guys think about this feature ? any feedback ?
>>>>>
>>>>> thanks,
>>>>>
>>>>> --
>>>>> regards,
>>>>>
>>>>> punith s
>>>>> cloudbyte.com
>>>>>
>>>>>
>>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Mike Tutkowski <mi...@solidfire.com>]

Hi,

Hypervisor rate limiting is only supported on KVM at the time being. You
are correct that it can get confusing if people use both hypervisor rate
limiting and then QoS of some flavor on the storage side, so CloudStack
only supports a user doing one or the other with a given volume.

As far as I know, we are still targeting March 14th for 4.4 Feature Freeze.
There has been some talk about extending this date a couple weeks, but - at
the moment - the 4.4 release manager (among others) are pushing to keep the
March 14th deadline.

I briefly investigated adding support for dynamically changing IOPS, but
that work was preempted by other CloudStack work I'm doing that was deemed
of higher priority. I do not think adding support to dynamically change
IOPS will be doable for CloudStack 4.4, but we should re-visit it in 4.5.
I'd be happy to work with you on that.

Talk to you later


On Wed, Mar 5, 2014 at 11:28 PM, Punith S <pu...@cloudbyte.com> wrote:

> yea mike i agree with you , i was talking with respect to changing iops of
> the datadisk on storage side, not with respect to hipervisor level settings,
>
> and like marcus suggested i tried to tie changing the disk iops with
> already available resize command but it has some limitations like i said
> before , that we need to detach the disk in order to do that which implies
> downtime for the user, but for a user who needs only to change the
> performance metric of the datadisk , he can change the iops by directly
> sending calls through the storage driver to the backend storage which
> supports change in Qos metrics like our cloudbyte elastistor and others
> without disturbing the hipervisor settings on fly with the available iops
> of the primary storage pool.
>
> and my doubt is that , how does hypervisor level throttling help in
> controlling the iops of the disk ?  and if so you might be wasting the
> available iops of the volume in backend!
> and thanks for the support mike , i have done some poc on this , we shall
> put this to 4.4 or 4.5,
> and i needed to push our cloudbyte elastistor storage plugin to 4.4 , so
> we should be pushing to master first before 28th of this month right ?
>
> thanks,
> punith
>
>
> On Thu, Mar 6, 2014 at 12:32 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Exactly...and that shouldn't be too much of a problem.
>>
>> The "trick" will come into play when we have to take into consideration
>> "generic" storage properties and that Min and Max IOPS should really be
>> part of this group.
>>
>>
>> On Wed, Mar 5, 2014 at 11:59 AM, Marcus <sh...@gmail.com> wrote:
>>
>> > Yes, that's what I mean. When the offering is changed, we need to have
>> > a hook in there that calls the applicable storage driver for the
>> > volume. Then the drivers can be free to implement or not implement.
>> >
>> > On Wed, Mar 5, 2014 at 11:36 AM, Mike Tutkowski
>> > <mi...@solidfire.com> wrote:
>> > > Hi Marcus,
>> > >
>> > > In 4.4 I'm adding the ability to allow admins to specify Min and Max
>> IOPS
>> > > for Compute Offerings, as well (support only for XenServer in 4.4).
>> > >
>> > > I agree changing of IOPS should be doable through changing a Disk or
>> > Compute
>> > > Offering, but this doesn't work right now as the storage framework
>> > doesn't
>> > > tell the plug-in in question about the change.
>> > >
>> > > This is part of what Punith should investigate as we move forward with
>> > this
>> > > in 4.5.
>> > >
>> > > Talk to you later,
>> > > Mike
>> > >
>> > >
>> > > On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
>> > >>
>> > >> Wouldn't this be implemented as just changing disk offerings? The
>> > >> resizeVolume API call already allows you to switch disk offerings, we
>> > >> just need to add a hook in there to optionally call the storage
>> driver
>> > >> (If volume is deployed to a primary storage) to make an update to the
>> > >> iops properties on the backend storage. Come to think of it,
>> depending
>> > >> on how storage drivers are implementing the iops/limits feature,
>> > >> resizeVolume might be breaking this, or simply requiring a reboot to
>> > >> apply. That is, if the storage driver is setting the iops just once
>> > >> upon volume creation, it's probably breaking when a user moves a disk
>> > >> between offerings that may have alternate iops limits (this is
>> > >> probably not the case for hypervisor throttling, as that's applied
>> > >> from whatever is current when the VM starts up).
>> > >>
>> > >> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
>> > >> <mi...@solidfire.com> wrote:
>> > >> > Hi,
>> > >> >
>> > >> > Perhaps I'm not following this correctly, but I'm a bit lost on
>> why we
>> > >> > are
>> > >> > talking about changing settings on running VMs.
>> > >> >
>> > >> > From what I understand, you are a representative of a storage
>> vendor
>> > >> > that
>> > >> > has a rate-limiting feature. You want to be able to not only set
>> the
>> > Max
>> > >> > IOPS, but also adjust them. Is this true?
>> > >> >
>> > >> > If so, I totally agree. SolidFire has control over Min and Max IOPS
>> > and
>> > >> > it
>> > >> > is on my to-do list to add support into CloudStack to be able to
>> > >> > dynamically change these values (right now customers do this from
>> the
>> > >> > SolidFire API or its GUI).
>> > >> >
>> > >> > If you would like to work on this feature, that would be great.
>> I'd be
>> > >> > happy to review your design and code.
>> > >> >
>> > >> > One complication is that we are looking at adding support for
>> generic
>> > >> > key/value pairs for storage plug-ins in 4.5 and this would
>> effectively
>> > >> > remove the need to have Min and Max IOPS as "special" fields in the
>> > >> > CloudStack API and GUI.
>> > >> >
>> > >> > I'm going to CC Chris Suichll (from NetApp) as he and I have
>> already
>> > >> > discussed this generic-properties concept. It would be good to get
>> his
>> > >> > feedback on how we might go about dynamically updating
>> storage-plug-in
>> > >> > key/value pairs.
>> > >> >
>> > >> > Thanks!
>> > >> > Mike
>> > >> >
>> > >> >
>> > >> > On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wido@widodh.nl
>> >
>> > >> > wrote:
>> > >> >
>> > >> >>
>> > >> >>
>> > >> >> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>> > >> >>
>> > >> >>> I was thinking about it last week.
>> > >> >>> AFAIK, libvirt-java 0.5.1 does not support change setting on
>> running
>> > >> >>> vms,
>> > >> >>> but virsh command line and libvirt API supports it.
>> > >> >>> so the sulution are
>> > >> >>> (1) change libvirt-java to support it, and make it released in
>> the
>> > >> >>> next
>> > >> >>> version. Maybe Wido can help us.
>> > >> >>>
>> > >> >>
>> > >> >> Sure! That seems the best way forward. What is currently lacking
>> in
>> > the
>> > >> >> libvirt-java bindings?
>> > >> >>
>> > >> >>
>> > >> >>  (2) call virsh command line.
>> > >> >>>
>> > >> >>>
>> > >> >> Please, please, do not do that. That's very hacky. We should
>> really
>> > >> >> keep
>> > >> >> using the libvirt-java bindings and stay away from invoking
>> binaries.
>> > >> >>
>> > >> >> Wido
>> > >> >>
>> > >> >>
>> > >> >>  -Wei
>> > >> >>>
>> > >> >>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>> > >> >>>
>> > >> >>>  hi guys,
>> > >> >>>>
>> > >> >>>> we are having a fixed max iops for each volume being attached to
>> > the
>> > >> >>>> instance in managed storage,
>> > >> >>>> so this a problem where we are making users to pre allocate the
>> > iops
>> > >> >>>> of
>> > >> >>>> the
>> > >> >>>> disk without having an option to change or resize it, similar to
>> > the
>> > >> >>>> size
>> > >> >>>> metric.
>> > >> >>>>
>> > >> >>>> so i would like to introduce a new feature which enables to
>> change
>> > or
>> > >> >>>> resize the volume iops on fly without detaching the datadisk of
>> the
>> > >> >>>> VM
>> > >> >>>> with
>> > >> >>>> zero downtime where performance of the datadisk can be altered
>> at
>> > any
>> > >> >>>> point
>> > >> >>>> with the available iops of the primary storage pool, which is
>> > similar
>> > >> >>>> in
>> > >> >>>> resizing the volume or datadisk of the vm , where in latter we
>> have
>> > >> >>>> to
>> > >> >>>> detach the datadisk.
>> > >> >>>>
>> > >> >>>> what do you guys think about this feature ? any feedback ?
>> > >> >>>>
>> > >> >>>> thanks,
>> > >> >>>>
>> > >> >>>> --
>> > >> >>>> regards,
>> > >> >>>>
>> > >> >>>> punith s
>> > >> >>>> cloudbyte.com
>> > >> >>>>
>> > >> >>>>
>> > >> >>>
>> > >> >
>> > >> >
>> > >> > --
>> > >> > *Mike Tutkowski*
>> > >> > *Senior CloudStack Developer, SolidFire Inc.*
>> > >> > e: mike.tutkowski@solidfire.com
>> > >> > o: 303.746.7302
>> > >> > Advancing the way the world uses the
>> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > >> > *(tm)*
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Mike Tutkowski
>> > > Senior CloudStack Developer, SolidFire Inc.
>> > > e: mike.tutkowski@solidfire.com
>> > > o: 303.746.7302
>> > > Advancing the way the world uses the cloud(tm)
>> >
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *(tm)*
>>
>
>
>
> --
> regards,
>
> punith s
> cloudbyte.com
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Punith S <pu...@cloudbyte.com>]

yea mike i agree with you , i was talking with respect to changing iops of
the datadisk on storage side, not with respect to hipervisor level settings,

and like marcus suggested i tried to tie changing the disk iops with
already available resize command but it has some limitations like i said
before , that we need to detach the disk in order to do that which implies
downtime for the user, but for a user who needs only to change the
performance metric of the datadisk , he can change the iops by directly
sending calls through the storage driver to the backend storage which
supports change in Qos metrics like our cloudbyte elastistor and others
without disturbing the hipervisor settings on fly with the available iops
of the primary storage pool.

and my doubt is that , how does hypervisor level throttling help in
controlling the iops of the disk ?  and if so you might be wasting the
available iops of the volume in backend!
and thanks for the support mike , i have done some poc on this , we shall
put this to 4.4 or 4.5,
and i needed to push our cloudbyte elastistor storage plugin to 4.4 , so we
should be pushing to master first before 28th of this month right ?

thanks,
punith


On Thu, Mar 6, 2014 at 12:32 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:

> Exactly...and that shouldn't be too much of a problem.
>
> The "trick" will come into play when we have to take into consideration
> "generic" storage properties and that Min and Max IOPS should really be
> part of this group.
>
>
> On Wed, Mar 5, 2014 at 11:59 AM, Marcus <sh...@gmail.com> wrote:
>
> > Yes, that's what I mean. When the offering is changed, we need to have
> > a hook in there that calls the applicable storage driver for the
> > volume. Then the drivers can be free to implement or not implement.
> >
> > On Wed, Mar 5, 2014 at 11:36 AM, Mike Tutkowski
> > <mi...@solidfire.com> wrote:
> > > Hi Marcus,
> > >
> > > In 4.4 I'm adding the ability to allow admins to specify Min and Max
> IOPS
> > > for Compute Offerings, as well (support only for XenServer in 4.4).
> > >
> > > I agree changing of IOPS should be doable through changing a Disk or
> > Compute
> > > Offering, but this doesn't work right now as the storage framework
> > doesn't
> > > tell the plug-in in question about the change.
> > >
> > > This is part of what Punith should investigate as we move forward with
> > this
> > > in 4.5.
> > >
> > > Talk to you later,
> > > Mike
> > >
> > >
> > > On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
> > >>
> > >> Wouldn't this be implemented as just changing disk offerings? The
> > >> resizeVolume API call already allows you to switch disk offerings, we
> > >> just need to add a hook in there to optionally call the storage driver
> > >> (If volume is deployed to a primary storage) to make an update to the
> > >> iops properties on the backend storage. Come to think of it, depending
> > >> on how storage drivers are implementing the iops/limits feature,
> > >> resizeVolume might be breaking this, or simply requiring a reboot to
> > >> apply. That is, if the storage driver is setting the iops just once
> > >> upon volume creation, it's probably breaking when a user moves a disk
> > >> between offerings that may have alternate iops limits (this is
> > >> probably not the case for hypervisor throttling, as that's applied
> > >> from whatever is current when the VM starts up).
> > >>
> > >> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
> > >> <mi...@solidfire.com> wrote:
> > >> > Hi,
> > >> >
> > >> > Perhaps I'm not following this correctly, but I'm a bit lost on why
> we
> > >> > are
> > >> > talking about changing settings on running VMs.
> > >> >
> > >> > From what I understand, you are a representative of a storage vendor
> > >> > that
> > >> > has a rate-limiting feature. You want to be able to not only set the
> > Max
> > >> > IOPS, but also adjust them. Is this true?
> > >> >
> > >> > If so, I totally agree. SolidFire has control over Min and Max IOPS
> > and
> > >> > it
> > >> > is on my to-do list to add support into CloudStack to be able to
> > >> > dynamically change these values (right now customers do this from
> the
> > >> > SolidFire API or its GUI).
> > >> >
> > >> > If you would like to work on this feature, that would be great. I'd
> be
> > >> > happy to review your design and code.
> > >> >
> > >> > One complication is that we are looking at adding support for
> generic
> > >> > key/value pairs for storage plug-ins in 4.5 and this would
> effectively
> > >> > remove the need to have Min and Max IOPS as "special" fields in the
> > >> > CloudStack API and GUI.
> > >> >
> > >> > I'm going to CC Chris Suichll (from NetApp) as he and I have already
> > >> > discussed this generic-properties concept. It would be good to get
> his
> > >> > feedback on how we might go about dynamically updating
> storage-plug-in
> > >> > key/value pairs.
> > >> >
> > >> > Thanks!
> > >> > Mike
> > >> >
> > >> >
> > >> > On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
> > >> > wrote:
> > >> >
> > >> >>
> > >> >>
> > >> >> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
> > >> >>
> > >> >>> I was thinking about it last week.
> > >> >>> AFAIK, libvirt-java 0.5.1 does not support change setting on
> running
> > >> >>> vms,
> > >> >>> but virsh command line and libvirt API supports it.
> > >> >>> so the sulution are
> > >> >>> (1) change libvirt-java to support it, and make it released in the
> > >> >>> next
> > >> >>> version. Maybe Wido can help us.
> > >> >>>
> > >> >>
> > >> >> Sure! That seems the best way forward. What is currently lacking in
> > the
> > >> >> libvirt-java bindings?
> > >> >>
> > >> >>
> > >> >>  (2) call virsh command line.
> > >> >>>
> > >> >>>
> > >> >> Please, please, do not do that. That's very hacky. We should really
> > >> >> keep
> > >> >> using the libvirt-java bindings and stay away from invoking
> binaries.
> > >> >>
> > >> >> Wido
> > >> >>
> > >> >>
> > >> >>  -Wei
> > >> >>>
> > >> >>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
> > >> >>>
> > >> >>>  hi guys,
> > >> >>>>
> > >> >>>> we are having a fixed max iops for each volume being attached to
> > the
> > >> >>>> instance in managed storage,
> > >> >>>> so this a problem where we are making users to pre allocate the
> > iops
> > >> >>>> of
> > >> >>>> the
> > >> >>>> disk without having an option to change or resize it, similar to
> > the
> > >> >>>> size
> > >> >>>> metric.
> > >> >>>>
> > >> >>>> so i would like to introduce a new feature which enables to
> change
> > or
> > >> >>>> resize the volume iops on fly without detaching the datadisk of
> the
> > >> >>>> VM
> > >> >>>> with
> > >> >>>> zero downtime where performance of the datadisk can be altered at
> > any
> > >> >>>> point
> > >> >>>> with the available iops of the primary storage pool, which is
> > similar
> > >> >>>> in
> > >> >>>> resizing the volume or datadisk of the vm , where in latter we
> have
> > >> >>>> to
> > >> >>>> detach the datadisk.
> > >> >>>>
> > >> >>>> what do you guys think about this feature ? any feedback ?
> > >> >>>>
> > >> >>>> thanks,
> > >> >>>>
> > >> >>>> --
> > >> >>>> regards,
> > >> >>>>
> > >> >>>> punith s
> > >> >>>> cloudbyte.com
> > >> >>>>
> > >> >>>>
> > >> >>>
> > >> >
> > >> >
> > >> > --
> > >> > *Mike Tutkowski*
> > >> > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > e: mike.tutkowski@solidfire.com
> > >> > o: 303.746.7302
> > >> > Advancing the way the world uses the
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > *(tm)*
> > >
> > >
> > >
> > >
> > > --
> > > Mike Tutkowski
> > > Senior CloudStack Developer, SolidFire Inc.
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the cloud(tm)
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
>



-- 
regards,

punith s
cloudbyte.com

Re: Change Volume IOPS on fly without detaching the disk feature.

[Mike Tutkowski <mi...@solidfire.com>]

Exactly...and that shouldn't be too much of a problem.

The "trick" will come into play when we have to take into consideration
"generic" storage properties and that Min and Max IOPS should really be
part of this group.


On Wed, Mar 5, 2014 at 11:59 AM, Marcus <sh...@gmail.com> wrote:

> Yes, that's what I mean. When the offering is changed, we need to have
> a hook in there that calls the applicable storage driver for the
> volume. Then the drivers can be free to implement or not implement.
>
> On Wed, Mar 5, 2014 at 11:36 AM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
> > Hi Marcus,
> >
> > In 4.4 I'm adding the ability to allow admins to specify Min and Max IOPS
> > for Compute Offerings, as well (support only for XenServer in 4.4).
> >
> > I agree changing of IOPS should be doable through changing a Disk or
> Compute
> > Offering, but this doesn't work right now as the storage framework
> doesn't
> > tell the plug-in in question about the change.
> >
> > This is part of what Punith should investigate as we move forward with
> this
> > in 4.5.
> >
> > Talk to you later,
> > Mike
> >
> >
> > On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
> >>
> >> Wouldn't this be implemented as just changing disk offerings? The
> >> resizeVolume API call already allows you to switch disk offerings, we
> >> just need to add a hook in there to optionally call the storage driver
> >> (If volume is deployed to a primary storage) to make an update to the
> >> iops properties on the backend storage. Come to think of it, depending
> >> on how storage drivers are implementing the iops/limits feature,
> >> resizeVolume might be breaking this, or simply requiring a reboot to
> >> apply. That is, if the storage driver is setting the iops just once
> >> upon volume creation, it's probably breaking when a user moves a disk
> >> between offerings that may have alternate iops limits (this is
> >> probably not the case for hypervisor throttling, as that's applied
> >> from whatever is current when the VM starts up).
> >>
> >> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
> >> <mi...@solidfire.com> wrote:
> >> > Hi,
> >> >
> >> > Perhaps I'm not following this correctly, but I'm a bit lost on why we
> >> > are
> >> > talking about changing settings on running VMs.
> >> >
> >> > From what I understand, you are a representative of a storage vendor
> >> > that
> >> > has a rate-limiting feature. You want to be able to not only set the
> Max
> >> > IOPS, but also adjust them. Is this true?
> >> >
> >> > If so, I totally agree. SolidFire has control over Min and Max IOPS
> and
> >> > it
> >> > is on my to-do list to add support into CloudStack to be able to
> >> > dynamically change these values (right now customers do this from the
> >> > SolidFire API or its GUI).
> >> >
> >> > If you would like to work on this feature, that would be great. I'd be
> >> > happy to review your design and code.
> >> >
> >> > One complication is that we are looking at adding support for generic
> >> > key/value pairs for storage plug-ins in 4.5 and this would effectively
> >> > remove the need to have Min and Max IOPS as "special" fields in the
> >> > CloudStack API and GUI.
> >> >
> >> > I'm going to CC Chris Suichll (from NetApp) as he and I have already
> >> > discussed this generic-properties concept. It would be good to get his
> >> > feedback on how we might go about dynamically updating storage-plug-in
> >> > key/value pairs.
> >> >
> >> > Thanks!
> >> > Mike
> >> >
> >> >
> >> > On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
> >> > wrote:
> >> >
> >> >>
> >> >>
> >> >> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
> >> >>
> >> >>> I was thinking about it last week.
> >> >>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
> >> >>> vms,
> >> >>> but virsh command line and libvirt API supports it.
> >> >>> so the sulution are
> >> >>> (1) change libvirt-java to support it, and make it released in the
> >> >>> next
> >> >>> version. Maybe Wido can help us.
> >> >>>
> >> >>
> >> >> Sure! That seems the best way forward. What is currently lacking in
> the
> >> >> libvirt-java bindings?
> >> >>
> >> >>
> >> >>  (2) call virsh command line.
> >> >>>
> >> >>>
> >> >> Please, please, do not do that. That's very hacky. We should really
> >> >> keep
> >> >> using the libvirt-java bindings and stay away from invoking binaries.
> >> >>
> >> >> Wido
> >> >>
> >> >>
> >> >>  -Wei
> >> >>>
> >> >>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
> >> >>>
> >> >>>  hi guys,
> >> >>>>
> >> >>>> we are having a fixed max iops for each volume being attached to
> the
> >> >>>> instance in managed storage,
> >> >>>> so this a problem where we are making users to pre allocate the
> iops
> >> >>>> of
> >> >>>> the
> >> >>>> disk without having an option to change or resize it, similar to
> the
> >> >>>> size
> >> >>>> metric.
> >> >>>>
> >> >>>> so i would like to introduce a new feature which enables to change
> or
> >> >>>> resize the volume iops on fly without detaching the datadisk of the
> >> >>>> VM
> >> >>>> with
> >> >>>> zero downtime where performance of the datadisk can be altered at
> any
> >> >>>> point
> >> >>>> with the available iops of the primary storage pool, which is
> similar
> >> >>>> in
> >> >>>> resizing the volume or datadisk of the vm , where in latter we have
> >> >>>> to
> >> >>>> detach the datadisk.
> >> >>>>
> >> >>>> what do you guys think about this feature ? any feedback ?
> >> >>>>
> >> >>>> thanks,
> >> >>>>
> >> >>>> --
> >> >>>> regards,
> >> >>>>
> >> >>>> punith s
> >> >>>> cloudbyte.com
> >> >>>>
> >> >>>>
> >> >>>
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *(tm)*
> >
> >
> >
> >
> > --
> > Mike Tutkowski
> > Senior CloudStack Developer, SolidFire Inc.
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud(tm)
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Marcus <sh...@gmail.com>]

Yes, that's what I mean. When the offering is changed, we need to have
a hook in there that calls the applicable storage driver for the
volume. Then the drivers can be free to implement or not implement.

On Wed, Mar 5, 2014 at 11:36 AM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> Hi Marcus,
>
> In 4.4 I'm adding the ability to allow admins to specify Min and Max IOPS
> for Compute Offerings, as well (support only for XenServer in 4.4).
>
> I agree changing of IOPS should be doable through changing a Disk or Compute
> Offering, but this doesn't work right now as the storage framework doesn't
> tell the plug-in in question about the change.
>
> This is part of what Punith should investigate as we move forward with this
> in 4.5.
>
> Talk to you later,
> Mike
>
>
> On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:
>>
>> Wouldn't this be implemented as just changing disk offerings? The
>> resizeVolume API call already allows you to switch disk offerings, we
>> just need to add a hook in there to optionally call the storage driver
>> (If volume is deployed to a primary storage) to make an update to the
>> iops properties on the backend storage. Come to think of it, depending
>> on how storage drivers are implementing the iops/limits feature,
>> resizeVolume might be breaking this, or simply requiring a reboot to
>> apply. That is, if the storage driver is setting the iops just once
>> upon volume creation, it's probably breaking when a user moves a disk
>> between offerings that may have alternate iops limits (this is
>> probably not the case for hypervisor throttling, as that's applied
>> from whatever is current when the VM starts up).
>>
>> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>> > Hi,
>> >
>> > Perhaps I'm not following this correctly, but I'm a bit lost on why we
>> > are
>> > talking about changing settings on running VMs.
>> >
>> > From what I understand, you are a representative of a storage vendor
>> > that
>> > has a rate-limiting feature. You want to be able to not only set the Max
>> > IOPS, but also adjust them. Is this true?
>> >
>> > If so, I totally agree. SolidFire has control over Min and Max IOPS and
>> > it
>> > is on my to-do list to add support into CloudStack to be able to
>> > dynamically change these values (right now customers do this from the
>> > SolidFire API or its GUI).
>> >
>> > If you would like to work on this feature, that would be great. I'd be
>> > happy to review your design and code.
>> >
>> > One complication is that we are looking at adding support for generic
>> > key/value pairs for storage plug-ins in 4.5 and this would effectively
>> > remove the need to have Min and Max IOPS as "special" fields in the
>> > CloudStack API and GUI.
>> >
>> > I'm going to CC Chris Suichll (from NetApp) as he and I have already
>> > discussed this generic-properties concept. It would be good to get his
>> > feedback on how we might go about dynamically updating storage-plug-in
>> > key/value pairs.
>> >
>> > Thanks!
>> > Mike
>> >
>> >
>> > On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
>> > wrote:
>> >
>> >>
>> >>
>> >> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>> >>
>> >>> I was thinking about it last week.
>> >>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
>> >>> vms,
>> >>> but virsh command line and libvirt API supports it.
>> >>> so the sulution are
>> >>> (1) change libvirt-java to support it, and make it released in the
>> >>> next
>> >>> version. Maybe Wido can help us.
>> >>>
>> >>
>> >> Sure! That seems the best way forward. What is currently lacking in the
>> >> libvirt-java bindings?
>> >>
>> >>
>> >>  (2) call virsh command line.
>> >>>
>> >>>
>> >> Please, please, do not do that. That's very hacky. We should really
>> >> keep
>> >> using the libvirt-java bindings and stay away from invoking binaries.
>> >>
>> >> Wido
>> >>
>> >>
>> >>  -Wei
>> >>>
>> >>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>> >>>
>> >>>  hi guys,
>> >>>>
>> >>>> we are having a fixed max iops for each volume being attached to the
>> >>>> instance in managed storage,
>> >>>> so this a problem where we are making users to pre allocate the iops
>> >>>> of
>> >>>> the
>> >>>> disk without having an option to change or resize it, similar to the
>> >>>> size
>> >>>> metric.
>> >>>>
>> >>>> so i would like to introduce a new feature which enables to change or
>> >>>> resize the volume iops on fly without detaching the datadisk of the
>> >>>> VM
>> >>>> with
>> >>>> zero downtime where performance of the datadisk can be altered at any
>> >>>> point
>> >>>> with the available iops of the primary storage pool, which is similar
>> >>>> in
>> >>>> resizing the volume or datadisk of the vm , where in latter we have
>> >>>> to
>> >>>> detach the datadisk.
>> >>>>
>> >>>> what do you guys think about this feature ? any feedback ?
>> >>>>
>> >>>> thanks,
>> >>>>
>> >>>> --
>> >>>> regards,
>> >>>>
>> >>>> punith s
>> >>>> cloudbyte.com
>> >>>>
>> >>>>
>> >>>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *(tm)*
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud(tm)

Re: Change Volume IOPS on fly without detaching the disk feature.

[Mike Tutkowski <mi...@solidfire.com>]

Hi Marcus,

In 4.4 I'm adding the ability to allow admins to specify Min and Max IOPS
for Compute Offerings, as well (support only for XenServer in 4.4).

I agree changing of IOPS should be doable through changing a Disk or
Compute Offering, but this doesn't work right now as the storage framework
doesn't tell the plug-in in question about the change.

This is part of what Punith should investigate as we move forward with this
in 4.5.

Talk to you later,
Mike


On Wed, Mar 5, 2014 at 11:12 AM, Marcus <sh...@gmail.com> wrote:

> Wouldn't this be implemented as just changing disk offerings? The
> resizeVolume API call already allows you to switch disk offerings, we
> just need to add a hook in there to optionally call the storage driver
> (If volume is deployed to a primary storage) to make an update to the
> iops properties on the backend storage. Come to think of it, depending
> on how storage drivers are implementing the iops/limits feature,
> resizeVolume might be breaking this, or simply requiring a reboot to
> apply. That is, if the storage driver is setting the iops just once
> upon volume creation, it's probably breaking when a user moves a disk
> between offerings that may have alternate iops limits (this is
> probably not the case for hypervisor throttling, as that's applied
> from whatever is current when the VM starts up).
>
> On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
> > Hi,
> >
> > Perhaps I'm not following this correctly, but I'm a bit lost on why we
> are
> > talking about changing settings on running VMs.
> >
> > From what I understand, you are a representative of a storage vendor that
> > has a rate-limiting feature. You want to be able to not only set the Max
> > IOPS, but also adjust them. Is this true?
> >
> > If so, I totally agree. SolidFire has control over Min and Max IOPS and
> it
> > is on my to-do list to add support into CloudStack to be able to
> > dynamically change these values (right now customers do this from the
> > SolidFire API or its GUI).
> >
> > If you would like to work on this feature, that would be great. I'd be
> > happy to review your design and code.
> >
> > One complication is that we are looking at adding support for generic
> > key/value pairs for storage plug-ins in 4.5 and this would effectively
> > remove the need to have Min and Max IOPS as "special" fields in the
> > CloudStack API and GUI.
> >
> > I'm going to CC Chris Suichll (from NetApp) as he and I have already
> > discussed this generic-properties concept. It would be good to get his
> > feedback on how we might go about dynamically updating storage-plug-in
> > key/value pairs.
> >
> > Thanks!
> > Mike
> >
> >
> > On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >
> >>
> >>
> >> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
> >>
> >>> I was thinking about it last week.
> >>> AFAIK, libvirt-java 0.5.1 does not support change setting on running
> vms,
> >>> but virsh command line and libvirt API supports it.
> >>> so the sulution are
> >>> (1) change libvirt-java to support it, and make it released in the next
> >>> version. Maybe Wido can help us.
> >>>
> >>
> >> Sure! That seems the best way forward. What is currently lacking in the
> >> libvirt-java bindings?
> >>
> >>
> >>  (2) call virsh command line.
> >>>
> >>>
> >> Please, please, do not do that. That's very hacky. We should really keep
> >> using the libvirt-java bindings and stay away from invoking binaries.
> >>
> >> Wido
> >>
> >>
> >>  -Wei
> >>>
> >>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
> >>>
> >>>  hi guys,
> >>>>
> >>>> we are having a fixed max iops for each volume being attached to the
> >>>> instance in managed storage,
> >>>> so this a problem where we are making users to pre allocate the iops
> of
> >>>> the
> >>>> disk without having an option to change or resize it, similar to the
> size
> >>>> metric.
> >>>>
> >>>> so i would like to introduce a new feature which enables to change or
> >>>> resize the volume iops on fly without detaching the datadisk of the VM
> >>>> with
> >>>> zero downtime where performance of the datadisk can be altered at any
> >>>> point
> >>>> with the available iops of the primary storage pool, which is similar
> in
> >>>> resizing the volume or datadisk of the vm , where in latter we have to
> >>>> detach the datadisk.
> >>>>
> >>>> what do you guys think about this feature ? any feedback ?
> >>>>
> >>>> thanks,
> >>>>
> >>>> --
> >>>> regards,
> >>>>
> >>>> punith s
> >>>> cloudbyte.com
> >>>>
> >>>>
> >>>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Marcus <sh...@gmail.com>]

Wouldn't this be implemented as just changing disk offerings? The
resizeVolume API call already allows you to switch disk offerings, we
just need to add a hook in there to optionally call the storage driver
(If volume is deployed to a primary storage) to make an update to the
iops properties on the backend storage. Come to think of it, depending
on how storage drivers are implementing the iops/limits feature,
resizeVolume might be breaking this, or simply requiring a reboot to
apply. That is, if the storage driver is setting the iops just once
upon volume creation, it's probably breaking when a user moves a disk
between offerings that may have alternate iops limits (this is
probably not the case for hypervisor throttling, as that's applied
from whatever is current when the VM starts up).

On Wed, Mar 5, 2014 at 9:58 AM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> Hi,
>
> Perhaps I'm not following this correctly, but I'm a bit lost on why we are
> talking about changing settings on running VMs.
>
> From what I understand, you are a representative of a storage vendor that
> has a rate-limiting feature. You want to be able to not only set the Max
> IOPS, but also adjust them. Is this true?
>
> If so, I totally agree. SolidFire has control over Min and Max IOPS and it
> is on my to-do list to add support into CloudStack to be able to
> dynamically change these values (right now customers do this from the
> SolidFire API or its GUI).
>
> If you would like to work on this feature, that would be great. I'd be
> happy to review your design and code.
>
> One complication is that we are looking at adding support for generic
> key/value pairs for storage plug-ins in 4.5 and this would effectively
> remove the need to have Min and Max IOPS as "special" fields in the
> CloudStack API and GUI.
>
> I'm going to CC Chris Suichll (from NetApp) as he and I have already
> discussed this generic-properties concept. It would be good to get his
> feedback on how we might go about dynamically updating storage-plug-in
> key/value pairs.
>
> Thanks!
> Mike
>
>
> On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>>
>>
>> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>>
>>> I was thinking about it last week.
>>> AFAIK, libvirt-java 0.5.1 does not support change setting on running vms,
>>> but virsh command line and libvirt API supports it.
>>> so the sulution are
>>> (1) change libvirt-java to support it, and make it released in the next
>>> version. Maybe Wido can help us.
>>>
>>
>> Sure! That seems the best way forward. What is currently lacking in the
>> libvirt-java bindings?
>>
>>
>>  (2) call virsh command line.
>>>
>>>
>> Please, please, do not do that. That's very hacky. We should really keep
>> using the libvirt-java bindings and stay away from invoking binaries.
>>
>> Wido
>>
>>
>>  -Wei
>>>
>>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>>
>>>  hi guys,
>>>>
>>>> we are having a fixed max iops for each volume being attached to the
>>>> instance in managed storage,
>>>> so this a problem where we are making users to pre allocate the iops of
>>>> the
>>>> disk without having an option to change or resize it, similar to the size
>>>> metric.
>>>>
>>>> so i would like to introduce a new feature which enables to change or
>>>> resize the volume iops on fly without detaching the datadisk of the VM
>>>> with
>>>> zero downtime where performance of the datadisk can be altered at any
>>>> point
>>>> with the available iops of the primary storage pool, which is similar in
>>>> resizing the volume or datadisk of the vm , where in latter we have to
>>>> detach the datadisk.
>>>>
>>>> what do you guys think about this feature ? any feedback ?
>>>>
>>>> thanks,
>>>>
>>>> --
>>>> regards,
>>>>
>>>> punith s
>>>> cloudbyte.com
>>>>
>>>>
>>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Mike Tutkowski <mi...@solidfire.com>]

Hi,

Perhaps I'm not following this correctly, but I'm a bit lost on why we are
talking about changing settings on running VMs.

>From what I understand, you are a representative of a storage vendor that
has a rate-limiting feature. You want to be able to not only set the Max
IOPS, but also adjust them. Is this true?

If so, I totally agree. SolidFire has control over Min and Max IOPS and it
is on my to-do list to add support into CloudStack to be able to
dynamically change these values (right now customers do this from the
SolidFire API or its GUI).

If you would like to work on this feature, that would be great. I'd be
happy to review your design and code.

One complication is that we are looking at adding support for generic
key/value pairs for storage plug-ins in 4.5 and this would effectively
remove the need to have Min and Max IOPS as "special" fields in the
CloudStack API and GUI.

I'm going to CC Chris Suichll (from NetApp) as he and I have already
discussed this generic-properties concept. It would be good to get his
feedback on how we might go about dynamically updating storage-plug-in
key/value pairs.

Thanks!
Mike


On Wed, Mar 5, 2014 at 3:12 AM, Wido den Hollander <wi...@widodh.nl> wrote:

>
>
> On 03/05/2014 10:12 AM, Wei ZHOU wrote:
>
>> I was thinking about it last week.
>> AFAIK, libvirt-java 0.5.1 does not support change setting on running vms,
>> but virsh command line and libvirt API supports it.
>> so the sulution are
>> (1) change libvirt-java to support it, and make it released in the next
>> version. Maybe Wido can help us.
>>
>
> Sure! That seems the best way forward. What is currently lacking in the
> libvirt-java bindings?
>
>
>  (2) call virsh command line.
>>
>>
> Please, please, do not do that. That's very hacky. We should really keep
> using the libvirt-java bindings and stay away from invoking binaries.
>
> Wido
>
>
>  -Wei
>>
>> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>>
>>  hi guys,
>>>
>>> we are having a fixed max iops for each volume being attached to the
>>> instance in managed storage,
>>> so this a problem where we are making users to pre allocate the iops of
>>> the
>>> disk without having an option to change or resize it, similar to the size
>>> metric.
>>>
>>> so i would like to introduce a new feature which enables to change or
>>> resize the volume iops on fly without detaching the datadisk of the VM
>>> with
>>> zero downtime where performance of the datadisk can be altered at any
>>> point
>>> with the available iops of the primary storage pool, which is similar in
>>> resizing the volume or datadisk of the vm , where in latter we have to
>>> detach the datadisk.
>>>
>>> what do you guys think about this feature ? any feedback ?
>>>
>>> thanks,
>>>
>>> --
>>> regards,
>>>
>>> punith s
>>> cloudbyte.com
>>>
>>>
>>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Re: Change Volume IOPS on fly without detaching the disk feature.

[Wido den Hollander <wi...@widodh.nl>]

On 03/05/2014 10:12 AM, Wei ZHOU wrote:
> I was thinking about it last week.
> AFAIK, libvirt-java 0.5.1 does not support change setting on running vms,
> but virsh command line and libvirt API supports it.
> so the sulution are
> (1) change libvirt-java to support it, and make it released in the next
> version. Maybe Wido can help us.

Sure! That seems the best way forward. What is currently lacking in the 
libvirt-java bindings?

> (2) call virsh command line.
>

Please, please, do not do that. That's very hacky. We should really keep 
using the libvirt-java bindings and stay away from invoking binaries.

Wido

> -Wei
>
> 2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:
>
>> hi guys,
>>
>> we are having a fixed max iops for each volume being attached to the
>> instance in managed storage,
>> so this a problem where we are making users to pre allocate the iops of the
>> disk without having an option to change or resize it, similar to the size
>> metric.
>>
>> so i would like to introduce a new feature which enables to change or
>> resize the volume iops on fly without detaching the datadisk of the VM with
>> zero downtime where performance of the datadisk can be altered at any point
>> with the available iops of the primary storage pool, which is similar in
>> resizing the volume or datadisk of the vm , where in latter we have to
>> detach the datadisk.
>>
>> what do you guys think about this feature ? any feedback ?
>>
>> thanks,
>>
>> --
>> regards,
>>
>> punith s
>> cloudbyte.com
>>
>

Re: Change Volume IOPS on fly without detaching the disk feature.

[Wei ZHOU <us...@gmail.com>]

I was thinking about it last week.
AFAIK, libvirt-java 0.5.1 does not support change setting on running vms,
but virsh command line and libvirt API supports it.
so the sulution are
(1) change libvirt-java to support it, and make it released in the next
version. Maybe Wido can help us.
(2) call virsh command line.

-Wei

2014-03-05 9:01 GMT+01:00 Punith S <pu...@cloudbyte.com>:

> hi guys,
>
> we are having a fixed max iops for each volume being attached to the
> instance in managed storage,
> so this a problem where we are making users to pre allocate the iops of the
> disk without having an option to change or resize it, similar to the size
> metric.
>
> so i would like to introduce a new feature which enables to change or
> resize the volume iops on fly without detaching the datadisk of the VM with
> zero downtime where performance of the datadisk can be altered at any point
> with the available iops of the primary storage pool, which is similar in
> resizing the volume or datadisk of the vm , where in latter we have to
> detach the datadisk.
>
> what do you guys think about this feature ? any feedback ?
>
> thanks,
>
> --
> regards,
>
> punith s
> cloudbyte.com
>